# Quantile-Based Estimation of Liu Parameter in the Linear Regression Model: Applications to Portland Cement and US Crime Data

Authors: Muhammad Suhail; Iqra Babar; Yousaf Ali Khan; Muhammad Imran; Zeeshan Nawaz
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

In multiple linear regression models, the multicollinearity problem mostly occurs when the explanatory variables are correlated among each other. It is well known that when the multicollinearity exists, the variance of the ordinary least square estimator is unstable. As a remedy, Liu in [1] developed a new method of estimation with biasing parameterd. In this paper, we have introduced a new method to estimate the biasing parameter in order to mitigate the problem of multicollinearity. The proposed method provides the class of estimators that are based on quantile of the regression coefficients. The performance of the new estimators is compared with the existing estimators through Monte Carlo simulation, where mean squared error and mean absolute error are considered as evaluation criteria of the estimators. Portland cement and US Crime data is used as an application to illustrate the benefit of the new estimators. Based on simulation and numerical study, it is concluded that the new estimators outperform the existing estimators in certain situations including high and severe cases of multicollinearity. 95% mean prediction interval of all the estimators is also computed for the Portland cement data. We recommend the use of new method to practitioners when the problem of high multicollinearity exists among the explanatory variables.


---

## Body

## 1. Introduction

The commonly used method of estimation in multiple linear regression models (MLRM) is the method of ordinary least squares (OLS) [1]. The results obtained through this method might be misleading when the problem of multicollinearity is present among the explanatory variables [2]. To overcome such problem, Ridge regression (RR) and Liu regression suggested by [1, 3], respectively, are the two widely used alternative methods. Liu estimator (LE) is usually preferred over RR, because it is the linear function of its biasing parameter d [4].The optimal value of biasing parameterd in Liu regression plays an important role in minimizing the variance. Many researchers have suggested several estimators for estimating d. Few of them are [4–6]. In this paper, the performance of some existing LEs is investigated, and a new method called as quantile based estimation of Liu or biasing parameter d is proposed. Also, the new estimators are compared with the existing ones through a Monte Carlo simulation based on mean squared error (MSE) and mean absolute error (MAE) performance criterions. The rest of the article is written as follows. The model estimation, newly proposed, and existing LEs are discussed in Section 2. The simulation design and results are discussed in Section 3. Section 4 includes two empirical applications to demonstrate the benefits of the new method. The conclusion of the paper is given in Section 5.


## 2. Statistical Methodology

Consider the following MLRM:(1)y=Xβ+ε,where y is the vector of random response variable of order n×1, X shows the fixed design matrix of explanatory variables with order n×p and β is the p+1×1 vector of population regression coefficients. ε is the vector of stochastic or random errors with order n×1 and is distributed as normal with mean Eε=0 and variance covariance matrix Eεε′=σ2In, In is an n×n identity matrix. The vector of OLS estimators for β is given below:(2)β^OLS=X′X−1X′Y.The OLS estimator is unbiased and more efficient than all other unbiased estimators [2]. However, in the presence of multicollinearity, OLS estimator becomes inefficient and provide large variance [4]. To circumvent such situation, numerous biased estimation methods are available, which provide smaller MSE than OLS, and LE is one of them. The Liu estimator defined by [1] is given as(3)β^LIU=X′X+I−1X′X+dIβ^OLS,0≤d≤1.In the presence of multicollinearity,β^LIU provides the smaller MSE than OLS [7]. The optimal choice of Liu parameter d plays a vital role in minimizing the MSE of β^LIU [8]. Some existing LEs for the biasing parameter d are given in the following subsection.


### 2.1. Some Existing Liu Estimators

Consider the canonical form of model (1):(4)y=Zα+ε,where Z=XD and α=α1,α2,…,αp′=D′β,D is an orthogonal matrix such that D′D=I and Z′Z=D′X′XD=Λ, Λ=diagλ1,λ2,…,λp consists of the eigenvalues of the X′X matrix. Note here that MSEα^=MSEβ^ so it suffices to consider the canonical form only. The OLS estimator can be defined in canonical form as follows:(5)α^OLS=Λ−1Z′y.The LE is defined as(6)α^LIU=Λ+I−1Λ+dIα^OLS,The first estimator ford was suggested by [1] and is given below:(7)d^j=α^j2−σ^2σ^2/λj+α^j2,where α^j is the jth element of α^, an OLS estimator of α. σ^2 is the unbiased estimator of population error variance σ2 and λj is the jth eigenvalue of the matrix X′X. Liu in [1] also suggested the following estimator:(8)D1=max0,absα^max2−σ^21/λmax+α^max2.Shukur et al. [6] considered the idea of [5, 9] and suggested the following four estimators:(9)D2=max0,medianα^j2−σ^21/λj+α^j2,D3=max0,1p∑j=1pα^j2−σ^21/λj+α^j2,D4=max0,maxα^j2−σ^21/λj+α^j2,D5=max0,minα^j2−σ^21/λj+α^j2.Based on the work of [10, 11], we proposed five new quantile based LEs, which are defined in the following section.


### 2.2. Proposed Method

Letd^1,d^2,…,d^p be the realized values of equation (7) and it can be written in the ascending order of magnitude as(10)d^1≤d^2≤⋯≤d^p,where d^1=mind^1,d^2,…,d^p and d^p=maxd^1,d^2,…,d^p. The setd^1,d^2,…,d^p is the order statistics for d^1,d^2,…,d^p and d^j,j=1,2,…,p, is the jth ordered observation. Now let d^γ,0<γ<1, be the 100γth quantile of d^1,d^2,…,d^p, and then the new proposed quantile estimator is(11)d^γ=d^jγ=α^j2γ−σ^2σ^2/λjγ+α^j2γ,such that(12)Pd^<dγ=γ,where “γ” is the quantile probability, α^j is the jth ordered element of α^, an OLS estimator of α, and λj is the jth ordered eigenvalue of the matrix X′X. The quantile probability generally depends on the degree of multicollinearity in order to obtain the minimum MSE or MAE. The new estimator d^γ defined in equation (11) depends on the quantile probability whose value is selected according to the degrees of multicollinearity. So the new proposed estimator is more robust to high or severe degrees of multicollinearity. Since the range of LE must be between zero and one, therefore we rewrite the equation (11) as(13)Dγ=max0,d^γ=max0,α^j2γ−σ^2σ^2/λjγ+α^j2γ.Equation (13) satisfies the interval condition for Liu parameter d suggested by [1]. In order to present the role of quantile probability, we choose some specific values for “γ” as: 0 (minimum), 0.25 (first quartile), 0.50 (median), 0.75 (third quartile) and 1 (maximum). The mathematical form of new LEs, denoted here by D6, D7, D8, D9, and D10, is given below:(14)D6=max0,d^0.0=max0,α^j20.0−σ^2σ^2/λj0.0+α^j20.0,D7=max0,d^0.25=max0,α^j20.25−σ^2σ^2/λj0.25+α^j20.25,D8=max0,d^0.50=max0,α^j20.50−σ^2σ^2/λj0.50+α^j20.50,D9=max0,d^0.75=max0,α^j20.75−σ^2σ^2/λj0.75+α^j20.75,D10=max0,d^1.0=max0,α^j21.0−σ^2σ^2/λj1.0+α^j21.0.


## 2.1. Some Existing Liu Estimators

Consider the canonical form of model (1):(4)y=Zα+ε,where Z=XD and α=α1,α2,…,αp′=D′β,D is an orthogonal matrix such that D′D=I and Z′Z=D′X′XD=Λ, Λ=diagλ1,λ2,…,λp consists of the eigenvalues of the X′X matrix. Note here that MSEα^=MSEβ^ so it suffices to consider the canonical form only. The OLS estimator can be defined in canonical form as follows:(5)α^OLS=Λ−1Z′y.The LE is defined as(6)α^LIU=Λ+I−1Λ+dIα^OLS,The first estimator ford was suggested by [1] and is given below:(7)d^j=α^j2−σ^2σ^2/λj+α^j2,where α^j is the jth element of α^, an OLS estimator of α. σ^2 is the unbiased estimator of population error variance σ2 and λj is the jth eigenvalue of the matrix X′X. Liu in [1] also suggested the following estimator:(8)D1=max0,absα^max2−σ^21/λmax+α^max2.Shukur et al. [6] considered the idea of [5, 9] and suggested the following four estimators:(9)D2=max0,medianα^j2−σ^21/λj+α^j2,D3=max0,1p∑j=1pα^j2−σ^21/λj+α^j2,D4=max0,maxα^j2−σ^21/λj+α^j2,D5=max0,minα^j2−σ^21/λj+α^j2.Based on the work of [10, 11], we proposed five new quantile based LEs, which are defined in the following section.


## 2.2. Proposed Method

Letd^1,d^2,…,d^p be the realized values of equation (7) and it can be written in the ascending order of magnitude as(10)d^1≤d^2≤⋯≤d^p,where d^1=mind^1,d^2,…,d^p and d^p=maxd^1,d^2,…,d^p. The setd^1,d^2,…,d^p is the order statistics for d^1,d^2,…,d^p and d^j,j=1,2,…,p, is the jth ordered observation. Now let d^γ,0<γ<1, be the 100γth quantile of d^1,d^2,…,d^p, and then the new proposed quantile estimator is(11)d^γ=d^jγ=α^j2γ−σ^2σ^2/λjγ+α^j2γ,such that(12)Pd^<dγ=γ,where “γ” is the quantile probability, α^j is the jth ordered element of α^, an OLS estimator of α, and λj is the jth ordered eigenvalue of the matrix X′X. The quantile probability generally depends on the degree of multicollinearity in order to obtain the minimum MSE or MAE. The new estimator d^γ defined in equation (11) depends on the quantile probability whose value is selected according to the degrees of multicollinearity. So the new proposed estimator is more robust to high or severe degrees of multicollinearity. Since the range of LE must be between zero and one, therefore we rewrite the equation (11) as(13)Dγ=max0,d^γ=max0,α^j2γ−σ^2σ^2/λjγ+α^j2γ.Equation (13) satisfies the interval condition for Liu parameter d suggested by [1]. In order to present the role of quantile probability, we choose some specific values for “γ” as: 0 (minimum), 0.25 (first quartile), 0.50 (median), 0.75 (third quartile) and 1 (maximum). The mathematical form of new LEs, denoted here by D6, D7, D8, D9, and D10, is given below:(14)D6=max0,d^0.0=max0,α^j20.0−σ^2σ^2/λj0.0+α^j20.0,D7=max0,d^0.25=max0,α^j20.25−σ^2σ^2/λj0.25+α^j20.25,D8=max0,d^0.50=max0,α^j20.50−σ^2σ^2/λj0.50+α^j20.50,D9=max0,d^0.75=max0,α^j20.75−σ^2σ^2/λj0.75+α^j20.75,D10=max0,d^1.0=max0,α^j21.0−σ^2σ^2/λj1.0+α^j21.0.


## 3. The Design of an Experiment

This section covers the Monte Carlo simulation experiment, performance criterion measures, and results and discussion.


### 3.1. The Monte Carlo Simulation

The performance of LEs is compared in this section through the simulation study. Following [12–14], the explanatory variables are generated as(15)xij=1−ρ21/2zij+ρzij+1,i=1,2,…,n,j=1,2,…,p,where ρ is the degree or level of multicollinearity between the explanatory variables, which are given as 0.80, 0.90, 0.99, and 0.999. zij are the random numbers obtained from the standard normal distribution.Then observations on the response variable are computed as(16)yi=β0+β1xi1+β2xi2+⋯+βpxip+εi,i=1,2,…,n,where εi∼N0,σ2, σ2 is the error variance. β0 is considered to be identically zero. Following [9], the eigenvector corresponding to maximum eigenvalue of the X′X matrix is taken as the vector of regression coefficients. Following [4–6], the different factors we choose to vary in our study are given below:Error variance:σ2=0.1,1,5,10Sample size:n=25,50,75,100,200,400Explanatory variables:p=4,8,16,32Multicollinearity:ρ=0.80,0.90,0.99,0.999


### 3.2. Performance Evaluation Criteria

Following [4], MSE and MAE criterions are used to judge the performance of the different LEs. Estimated MSE (EMSE) and MAE (EMAE) are defined as(17)MSE=∑i=1Mβ^i−β′β^i−βM,MAE=∑i=1Mβ^i−βM,where βi^ is the estimated value of β. M shows the simulation runs. In this study, we choose M = 5000. The EMSE simulation results are presented in Tables 1–6 and EMAE in Tables 7, 8 . The results are discussed in the following section.Table 1
EMSE withϵ∼N0,1 and p=4.
n2550ρ0.800.900.990.9990.800.900.990.999OLS0.86471.764217.9023169.98370.33960.75728.534289.7533D10.58081.113415.4244166.54020.27600.51756.491086.3338D20.52420.68650.84201.54610.27750.47020.71270.5065D30.53420.70471.08373.23590.27800.47230.81170.9502D40.63430.86772.957121.73470.28970.50081.33895.0789D50.46440.60170.56890.25610.26770.44920.62690.1773D60.47140.59330.52740.11050.27120.45350.61500.1360D70.50560.63160.54800.18760.27780.46400.61960.1625D80.51040.63400.67840.93970.27870.46200.65000.3115D90.53530.821811.5467158.74300.28670.49824.244879.6463D100.58081.113415.4244166.54020.27600.51756.491086.3338n75100OLS0.18570.38673.989641.57070.11480.24162.587826.2797D10.16290.29642.704238.54420.10510.19941.656623.5315D20.16570.29510.70720.43580.10700.20160.64370.4751D30.16600.29500.74850.63180.10730.20160.66640.5933D40.17170.30610.93092.04860.11030.20810.76251.3133D50.16110.28370.67650.29650.10480.19530.62590.3961D60.16250.28630.66980.27900.10540.19650.62050.3856D70.16670.29460.67580.29020.10780.20220.62870.3924D80.16790.29480.67860.35150.10870.20300.62650.4277D90.17000.30671.676532.92590.10920.20901.073318.8356D100.16290.29642.704238.54420.10510.19941.656623.5315Bold values indicate the minimum MSE.Table 2
EMSE withɛ∼N0,1 and p=8.
n2550ρ0.800.900.990.9990.800.900.990.999OLS4.466110.0978111.22831110.99400.66211.340713.5747137.7026D13.26428.2668107.83161107.12160.53400.959411.0166133.4576D21.40051.59671.48732.80120.54350.85991.67390.7777D31.50541.74222.34418.70560.54640.86341.79281.2676D42.68623.275717.9696158.08720.59240.94382.82948.7950D51.03411.26241.09260.26780.50830.79911.59320.5333D61.00151.22461.06020.21670.51300.80191.58770.5213D71.18551.36961.08630.27440.53480.84161.61620.5393D81.41201.49511.27281.80720.55570.86911.61410.6823D91.34681.54465.134139.85630.55200.85411.74141.6307D103.26428.2668107.83161107.12160.53400.959411.0166133.4576n75100OLS0.53871.132412.0519121.23890.43500.87928.895589.0391D10.42800.79869.7917117.26420.35960.64256.848485.0153D20.43860.70901.61200.78090.36920.60861.61440.8232D30.43890.71041.68291.10800.36700.60611.66161.0510D40.47070.76562.30315.75380.38060.62932.05613.7974D50.40860.66031.56260.64250.35030.57791.58020.7294D60.41120.66111.55650.63410.35210.57951.57700.7237D70.43400.70031.58600.64700.36740.60541.60290.7332D80.44450.71481.57670.72600.37100.61051.59300.7895D90.44570.70761.61991.14890.36980.60401.60371.0074D100.42800.79869.7917117.26420.35960.64256.848485.0153Bold values indicate the minimum MSE.Table 3
EMSE withɛ∼N0,5 and p=4.
n2550ρ0.800.900.990.9990.800.900.990.999OLS41.464242.9732454.50904496.538818.229018.9241211.71162227.7276D126.176039.7862403.05334426.980513.370923.2564170.54872157.6404D2137.0247174.2478188.24201675.730120.785137.173866.7591492.4123D3378.05221374.531439523.4204490286.330536.1051153.79168463.1000129718.4494D42975.171915340.5083594331.06327470388.8610168.11681275.1605123401.63911962914.4830D515.753323.326757.2993306.00848.317113.868228.059474.3255D632.834037.909822.64654.195411.776519.935522.88804.7488D782.1737109.063854.0141156.543517.127531.190430.491640.7833D8104.7290129.576993.66941064.416918.640734.281737.5308262.5660D942.672053.2101348.49404338.941518.077834.8008140.49002082.4728D1026.176039.7862403.05334426.980513.370923.2564170.54872157.6404n75100OLS6.47709.114199.90041013.32134.84995.877962.7563665.4290D17.784914.669277.4999950.86004.908010.316550.7675608.5832D210.729020.996442.3556174.17926.339213.607234.408992.2086D312.957941.99182191.749741734.77376.607719.7573996.461621399.1122D435.4892219.476729783.5059627203.799012.856771.308412681.4377319880.7932D54.80907.952625.350927.45563.13615.545521.040019.9151D65.865610.117822.33798.50723.59016.564419.036710.9167D78.766516.889233.603419.30005.250010.820930.175616.6966D89.560719.142432.533984.00755.650612.320030.739045.0526D99.570021.111768.9550883.00605.611313.665453.8102547.8611D107.784914.669277.4999950.86004.908010.316550.7675608.5832Bold values indicate the minimum MSE.Table 4
EMSE withɛ∼N0,5 and p=8.
n2550ρ0.800.900.990.9990.800.900.990.999OLS113.7344248.42632749.931827301.909226.421032.8776338.04253442.7954D191.8201208.62142672.618927216.274221.758232.8855279.03883346.8484D21062.0091773.0738646.71005747.929154.839389.7872111.6563382.6700D34249.44558958.253894691.2091889078.767574.4252196.15023581.314855459.1032D4103592.5974346235.24775504480.475152208065.6888465.96652876.8675177596.77823166224.5262D548.298763.858972.7158188.986415.073523.284250.151127.8849D645.003943.332929.56395.758920.338030.237748.801815.3856D7509.4831426.537585.5024186.347441.485571.136384.733336.4997D81111.9509860.3599261.27283877.213256.553095.596078.9714236.9419D9510.7864355.22021286.355316591.893044.599971.285897.63261105.5173D1091.8201208.62142672.618927216.274221.758232.8855279.03883346.8484n75100OLS23.680428.8332293.20242994.066620.948922.2814228.85292206.8140D119.379430.4080240.82662903.681816.468226.3007181.93242116.7986D247.536278.476188.1685214.792227.642346.439272.6255126.2054D362.5967164.41742542.103934753.513534.416684.53151510.197422352.1191D4354.78122201.6875124050.88581975140.4297169.1426951.977469486.01271269566.1336D511.750919.516346.648623.80219.866016.300244.891824.1901D614.406922.393843.483717.265511.355918.369043.519919.7012D738.555466.207477.823829.566925.499643.416970.510228.8082D846.333380.236971.0713130.205327.042246.712364.515883.4833D942.972671.183868.5353629.686124.574040.867556.6078323.1200D1019.379430.4080240.82662903.681816.468226.3007181.93242116.7986Bold values indicate the minimum MSE.Table 5
EMSE withɛ∼N0,0.1.
n200400p16ρ0.800.900.990.9990.800.900.990.999OLS0.00350.00750.07930.80210.00190.00390.04100.4163D10.00350.00730.07360.72260.00190.00390.03860.3744D20.00310.00560.02470.02750.00170.00330.01670.0343D30.00310.00570.02630.03920.00180.00330.01750.0416D40.00350.00730.07360.71030.00190.00390.03860.3728D50.00310.00560.02470.02750.00170.00330.01670.0343D60.00310.00560.02470.02750.00170.00330.01670.0343D70.00310.00560.02470.02750.00170.00330.01670.0343D80.00310.00560.02470.02750.00170.00330.01670.0343D90.00310.00560.02470.02750.00170.00330.01670.0343D100.00350.00730.07360.72260.00190.00390.03860.3744p32OLS0.00990.02130.21972.18190.00390.00800.08220.8277D10.00970.02070.20882.08730.00380.00790.07910.7841D20.00730.01220.05500.05440.00340.00610.03230.0730D30.00740.01240.05680.06670.00340.00610.03300.0798D40.00970.02070.20882.02840.00380.00790.07910.7765D50.00730.01220.05500.05440.00340.00610.03230.0730D60.00730.01220.05500.05440.00340.00610.03230.0730D70.00730.01220.05500.05440.00340.00610.03230.0730D80.00730.01220.05500.05440.00340.00610.03230.0730D90.00730.01220.05500.05440.00340.00610.03230.0730D100.00970.02070.20882.08730.00380.00790.07910.7841Table 6
EMSE withɛ∼N0,10.
n200400p16ρ0.800.900.990.9990.800.900.990.999OLS35.307374.6252788.65478050.311618.792839.7571409.07044084.6535D158.774599.4727627.27267630.194736.094267.2299331.97483737.2599D2330.6592776.50511029.5021871.0269105.0624257.7099608.7140579.6858D3410.90691320.703620406.3241321323.4893117.7264367.64357379.0438119656.0579D43950.177434835.89673510745.999275009924.6589696.53496218.58171113803.019127530620.4314D536.900264.4541291.8689320.637419.693336.4964200.5263365.5992D647.415184.1486283.5081297.120621.861841.6713182.9172356.2603D7304.0021719.71001131.0888501.289495.5070233.0943605.4684524.1439D8337.4555819.17391101.8653505.4253109.5028274.5622662.2097444.3032D9273.7871640.8076674.27691735.715196.5125240.9880515.3343613.2459D1058.774599.4727627.27267630.194736.094267.2299331.97483737.2599p32OLS99.4473209.85152186.335022167.582338.456880.6980810.67388312.4423D1126.8498204.54741926.103421620.915665.4947110.6764676.04457859.4889D22962.50194694.96562240.28091777.2502417.3801990.87541302.90781189.0352D33901.46527511.530823463.7648234740.7023471.16711287.30197116.482267892.5484D454753.9636314083.215512238618.7632201244100.95713495.355929928.22602792922.933156051950.0538D582.4224146.5039582.2643583.835337.053966.1789367.3434756.0658D6103.4789156.2744569.0643558.900441.319173.8115335.2977747.2898D72545.60924065.91922422.9089921.0806362.5921851.43431237.22371099.3145D83486.90555656.67052660.4023973.9301453.95941086.77201448.3273922.6883D92728.54504209.19451468.15164644.7662439.10821067.40701219.62571188.9104D10126.8498204.54741926.103421620.915665.4947110.6764676.04457859.4889Table 7
EMAE withɛ∼N0,1 and p=4.
n2550ρ0.850.950.990.9990.850.950.990.999OLS1.43822.04046.420119.85980.91551.35424.540214.6991D11.06411.38224.723516.18450.76101.00813.178112.1143D21.03851.19611.27461.41090.76340.98071.19830.9127D31.04671.20901.45122.06720.76420.98261.28301.2601D41.12051.31412.16834.79160.77741.00731.59702.6147D50.98921.12981.08730.65540.75240.96171.13840.6004D60.99641.12921.04820.49180.75590.96571.12930.5412D71.02601.16071.06840.59860.76380.97571.13340.5802D81.03701.16651.16481.10600.76680.97641.15430.7462D91.06111.27613.918315.58640.77971.01662.508311.4210D101.06411.38224.723516.18450.76101.00813.178112.1143n75100OLS0.67470.96653.05779.88980.53130.76102.45577.8329D10.58210.76821.98537.79470.46970.62491.56875.9675D20.58630.76861.19980.90610.47320.62801.14160.9495D30.58680.76851.23441.09590.47360.62811.16001.0734D40.59470.78011.36011.80000.47880.63591.23041.5178D50.58000.75681.17770.77170.46910.62051.12670.8766D60.58170.75911.17370.74980.47030.62171.12410.8647D70.58770.76821.17950.76340.47450.62881.13160.8719D80.59030.77011.18070.82800.47650.63091.12990.9061D90.59490.78851.63466.96040.47780.64141.36345.1163D100.58210.76821.98537.79470.46970.62491.56875.9675Bold values indicate the minimum MAE.Table 8
EMAE withɛ∼N0,1 and p=8.
n2550ρ0.850.950.990.9990.850.950.990.999OLS4.19066.120519.750362.49921.75932.45567.725024.4799D13.00254.498416.110652.38291.50641.96796.115321.7759D22.39712.64312.54342.68821.52121.92152.74981.8304D32.46152.73783.10694.63261.52451.92442.84522.3196D42.98823.39616.490016.61391.57291.99233.46695.2519D52.13712.38512.16071.05171.48241.86542.68141.5286D62.11882.35392.11300.94231.48711.86952.67711.5090D72.26612.48842.15341.07391.51171.90562.70411.5362D82.41272.58292.33842.27301.53481.93022.70101.7221D92.37262.60124.03928.98551.53521.92252.78932.5095D103.00254.498416.110652.38291.50641.96796.115321.7759N75100OLS1.54762.20207.042122.27631.40111.97556.209119.5233D11.32141.75625.508119.65491.21961.61624.758117.2197D21.33691.71912.69361.84121.23251.60312.69681.9021D31.33731.72022.75412.19511.23001.60072.73652.1581D41.37021.76683.16704.33491.24591.62273.02163.7254D51.30431.67412.64901.66601.21001.57252.66631.7887D61.30681.67602.64361.65301.21171.57442.66401.7807D71.33201.71162.67241.67161.23041.60002.68801.7933D81.34331.72452.66381.77371.23471.60512.67951.8615D91.34821.72222.69312.18981.23491.60122.68462.0897D101.32141.75625.508119.65491.21961.61624.758117.2197Bold values indicate the minimum MAE.


### 3.3. Results and Discussion

The EMSE and EMAE values of the new and existing Liu estimators are presented in Tables1–6 and 7, 8 respectively. The performance of the LEs is evaluated by varying the values of factors such as multicollinearity, error variance, sample size, and the explanatory variables. A general remark from the literature is that these factors have the significant effect on the simulation design. The effect of each factor on EMSE and EMAE of estimators is discussed below:The first factor we considered is the effect of multicollinearity on the EMSE and EMAE of estimators. The EMSE and EMAE of all the estimators increase by increasing the degree of multicollinearity generally except D5–D8. Estimators D5–D8 first increase when multicollinearity increases from mild to high and decrease for severe multicollinearity. It is evident from these tables that the LE is always superior to the OLS estimator. For mild multicollinearity (ρ = 0.80), D5 outperforms others. Estimators D6 and D7 are the close competitors to D5. When the degree of multicollinearity is considered to be high (ρ = 0.99) or severe (ρ = 0.999), then the proposed estimators D6 and D7 outperform generally. However, the performance of D6 is good among others, because it yields lowest EMSE and EMAE when the degree of multicollinearity is high or severe.Secondly, the effect of sample size on the EMSE and EMAE of estimators is considered. In general, the EMSE and EMAE of OLS and LE decrease by raising the sample size. The proposed estimators D6 and D7 exhibit lowest EMSE in most of the cases, while D5 remains the closest competitor to the proposed estimators.In the third case, we varied the number of explanatory variables from 4 to 8. It is found that EMSE and EMAE of all the estimators increase. But the performance pattern remains the same as in the case of multicollinearity and sample size. It is seen that the increase in the EMSE of OLS estimator is relatively higher than all LEs. LE with Liu parameter D6 exhibits lowest EMSE and EMAE for both the cases of explanatory variables.Finally, the performance of estimators is compared by varying error variance. When the error variance increases from one to five, the EMSE of all the estimators deteriorates, except the new estimator D6, which is least affected.The concluded remarks from Tables1–8 are that the new estimators D6 and D7 have shown better performance than the other existing LEs in terms of smaller EMSE and EMAE, in situations of high multicollinearity disregarding the values of other factors. The proposed estimators also outperform the OLS estimator substantially. Therefore, based on EMSE and EMAE criterion, we can infer that the new estimator D6 is more efficient and is the best choice for the practitioners in the presence of high and severe multicollinearity.


## 3.1. The Monte Carlo Simulation

The performance of LEs is compared in this section through the simulation study. Following [12–14], the explanatory variables are generated as(15)xij=1−ρ21/2zij+ρzij+1,i=1,2,…,n,j=1,2,…,p,where ρ is the degree or level of multicollinearity between the explanatory variables, which are given as 0.80, 0.90, 0.99, and 0.999. zij are the random numbers obtained from the standard normal distribution.Then observations on the response variable are computed as(16)yi=β0+β1xi1+β2xi2+⋯+βpxip+εi,i=1,2,…,n,where εi∼N0,σ2, σ2 is the error variance. β0 is considered to be identically zero. Following [9], the eigenvector corresponding to maximum eigenvalue of the X′X matrix is taken as the vector of regression coefficients. Following [4–6], the different factors we choose to vary in our study are given below:Error variance:σ2=0.1,1,5,10Sample size:n=25,50,75,100,200,400Explanatory variables:p=4,8,16,32Multicollinearity:ρ=0.80,0.90,0.99,0.999


## 3.2. Performance Evaluation Criteria

Following [4], MSE and MAE criterions are used to judge the performance of the different LEs. Estimated MSE (EMSE) and MAE (EMAE) are defined as(17)MSE=∑i=1Mβ^i−β′β^i−βM,MAE=∑i=1Mβ^i−βM,where βi^ is the estimated value of β. M shows the simulation runs. In this study, we choose M = 5000. The EMSE simulation results are presented in Tables 1–6 and EMAE in Tables 7, 8 . The results are discussed in the following section.Table 1
EMSE withϵ∼N0,1 and p=4.
n2550ρ0.800.900.990.9990.800.900.990.999OLS0.86471.764217.9023169.98370.33960.75728.534289.7533D10.58081.113415.4244166.54020.27600.51756.491086.3338D20.52420.68650.84201.54610.27750.47020.71270.5065D30.53420.70471.08373.23590.27800.47230.81170.9502D40.63430.86772.957121.73470.28970.50081.33895.0789D50.46440.60170.56890.25610.26770.44920.62690.1773D60.47140.59330.52740.11050.27120.45350.61500.1360D70.50560.63160.54800.18760.27780.46400.61960.1625D80.51040.63400.67840.93970.27870.46200.65000.3115D90.53530.821811.5467158.74300.28670.49824.244879.6463D100.58081.113415.4244166.54020.27600.51756.491086.3338n75100OLS0.18570.38673.989641.57070.11480.24162.587826.2797D10.16290.29642.704238.54420.10510.19941.656623.5315D20.16570.29510.70720.43580.10700.20160.64370.4751D30.16600.29500.74850.63180.10730.20160.66640.5933D40.17170.30610.93092.04860.11030.20810.76251.3133D50.16110.28370.67650.29650.10480.19530.62590.3961D60.16250.28630.66980.27900.10540.19650.62050.3856D70.16670.29460.67580.29020.10780.20220.62870.3924D80.16790.29480.67860.35150.10870.20300.62650.4277D90.17000.30671.676532.92590.10920.20901.073318.8356D100.16290.29642.704238.54420.10510.19941.656623.5315Bold values indicate the minimum MSE.Table 2
EMSE withɛ∼N0,1 and p=8.
n2550ρ0.800.900.990.9990.800.900.990.999OLS4.466110.0978111.22831110.99400.66211.340713.5747137.7026D13.26428.2668107.83161107.12160.53400.959411.0166133.4576D21.40051.59671.48732.80120.54350.85991.67390.7777D31.50541.74222.34418.70560.54640.86341.79281.2676D42.68623.275717.9696158.08720.59240.94382.82948.7950D51.03411.26241.09260.26780.50830.79911.59320.5333D61.00151.22461.06020.21670.51300.80191.58770.5213D71.18551.36961.08630.27440.53480.84161.61620.5393D81.41201.49511.27281.80720.55570.86911.61410.6823D91.34681.54465.134139.85630.55200.85411.74141.6307D103.26428.2668107.83161107.12160.53400.959411.0166133.4576n75100OLS0.53871.132412.0519121.23890.43500.87928.895589.0391D10.42800.79869.7917117.26420.35960.64256.848485.0153D20.43860.70901.61200.78090.36920.60861.61440.8232D30.43890.71041.68291.10800.36700.60611.66161.0510D40.47070.76562.30315.75380.38060.62932.05613.7974D50.40860.66031.56260.64250.35030.57791.58020.7294D60.41120.66111.55650.63410.35210.57951.57700.7237D70.43400.70031.58600.64700.36740.60541.60290.7332D80.44450.71481.57670.72600.37100.61051.59300.7895D90.44570.70761.61991.14890.36980.60401.60371.0074D100.42800.79869.7917117.26420.35960.64256.848485.0153Bold values indicate the minimum MSE.Table 3
EMSE withɛ∼N0,5 and p=4.
n2550ρ0.800.900.990.9990.800.900.990.999OLS41.464242.9732454.50904496.538818.229018.9241211.71162227.7276D126.176039.7862403.05334426.980513.370923.2564170.54872157.6404D2137.0247174.2478188.24201675.730120.785137.173866.7591492.4123D3378.05221374.531439523.4204490286.330536.1051153.79168463.1000129718.4494D42975.171915340.5083594331.06327470388.8610168.11681275.1605123401.63911962914.4830D515.753323.326757.2993306.00848.317113.868228.059474.3255D632.834037.909822.64654.195411.776519.935522.88804.7488D782.1737109.063854.0141156.543517.127531.190430.491640.7833D8104.7290129.576993.66941064.416918.640734.281737.5308262.5660D942.672053.2101348.49404338.941518.077834.8008140.49002082.4728D1026.176039.7862403.05334426.980513.370923.2564170.54872157.6404n75100OLS6.47709.114199.90041013.32134.84995.877962.7563665.4290D17.784914.669277.4999950.86004.908010.316550.7675608.5832D210.729020.996442.3556174.17926.339213.607234.408992.2086D312.957941.99182191.749741734.77376.607719.7573996.461621399.1122D435.4892219.476729783.5059627203.799012.856771.308412681.4377319880.7932D54.80907.952625.350927.45563.13615.545521.040019.9151D65.865610.117822.33798.50723.59016.564419.036710.9167D78.766516.889233.603419.30005.250010.820930.175616.6966D89.560719.142432.533984.00755.650612.320030.739045.0526D99.570021.111768.9550883.00605.611313.665453.8102547.8611D107.784914.669277.4999950.86004.908010.316550.7675608.5832Bold values indicate the minimum MSE.Table 4
EMSE withɛ∼N0,5 and p=8.
n2550ρ0.800.900.990.9990.800.900.990.999OLS113.7344248.42632749.931827301.909226.421032.8776338.04253442.7954D191.8201208.62142672.618927216.274221.758232.8855279.03883346.8484D21062.0091773.0738646.71005747.929154.839389.7872111.6563382.6700D34249.44558958.253894691.2091889078.767574.4252196.15023581.314855459.1032D4103592.5974346235.24775504480.475152208065.6888465.96652876.8675177596.77823166224.5262D548.298763.858972.7158188.986415.073523.284250.151127.8849D645.003943.332929.56395.758920.338030.237748.801815.3856D7509.4831426.537585.5024186.347441.485571.136384.733336.4997D81111.9509860.3599261.27283877.213256.553095.596078.9714236.9419D9510.7864355.22021286.355316591.893044.599971.285897.63261105.5173D1091.8201208.62142672.618927216.274221.758232.8855279.03883346.8484n75100OLS23.680428.8332293.20242994.066620.948922.2814228.85292206.8140D119.379430.4080240.82662903.681816.468226.3007181.93242116.7986D247.536278.476188.1685214.792227.642346.439272.6255126.2054D362.5967164.41742542.103934753.513534.416684.53151510.197422352.1191D4354.78122201.6875124050.88581975140.4297169.1426951.977469486.01271269566.1336D511.750919.516346.648623.80219.866016.300244.891824.1901D614.406922.393843.483717.265511.355918.369043.519919.7012D738.555466.207477.823829.566925.499643.416970.510228.8082D846.333380.236971.0713130.205327.042246.712364.515883.4833D942.972671.183868.5353629.686124.574040.867556.6078323.1200D1019.379430.4080240.82662903.681816.468226.3007181.93242116.7986Bold values indicate the minimum MSE.Table 5
EMSE withɛ∼N0,0.1.
n200400p16ρ0.800.900.990.9990.800.900.990.999OLS0.00350.00750.07930.80210.00190.00390.04100.4163D10.00350.00730.07360.72260.00190.00390.03860.3744D20.00310.00560.02470.02750.00170.00330.01670.0343D30.00310.00570.02630.03920.00180.00330.01750.0416D40.00350.00730.07360.71030.00190.00390.03860.3728D50.00310.00560.02470.02750.00170.00330.01670.0343D60.00310.00560.02470.02750.00170.00330.01670.0343D70.00310.00560.02470.02750.00170.00330.01670.0343D80.00310.00560.02470.02750.00170.00330.01670.0343D90.00310.00560.02470.02750.00170.00330.01670.0343D100.00350.00730.07360.72260.00190.00390.03860.3744p32OLS0.00990.02130.21972.18190.00390.00800.08220.8277D10.00970.02070.20882.08730.00380.00790.07910.7841D20.00730.01220.05500.05440.00340.00610.03230.0730D30.00740.01240.05680.06670.00340.00610.03300.0798D40.00970.02070.20882.02840.00380.00790.07910.7765D50.00730.01220.05500.05440.00340.00610.03230.0730D60.00730.01220.05500.05440.00340.00610.03230.0730D70.00730.01220.05500.05440.00340.00610.03230.0730D80.00730.01220.05500.05440.00340.00610.03230.0730D90.00730.01220.05500.05440.00340.00610.03230.0730D100.00970.02070.20882.08730.00380.00790.07910.7841Table 6
EMSE withɛ∼N0,10.
n200400p16ρ0.800.900.990.9990.800.900.990.999OLS35.307374.6252788.65478050.311618.792839.7571409.07044084.6535D158.774599.4727627.27267630.194736.094267.2299331.97483737.2599D2330.6592776.50511029.5021871.0269105.0624257.7099608.7140579.6858D3410.90691320.703620406.3241321323.4893117.7264367.64357379.0438119656.0579D43950.177434835.89673510745.999275009924.6589696.53496218.58171113803.019127530620.4314D536.900264.4541291.8689320.637419.693336.4964200.5263365.5992D647.415184.1486283.5081297.120621.861841.6713182.9172356.2603D7304.0021719.71001131.0888501.289495.5070233.0943605.4684524.1439D8337.4555819.17391101.8653505.4253109.5028274.5622662.2097444.3032D9273.7871640.8076674.27691735.715196.5125240.9880515.3343613.2459D1058.774599.4727627.27267630.194736.094267.2299331.97483737.2599p32OLS99.4473209.85152186.335022167.582338.456880.6980810.67388312.4423D1126.8498204.54741926.103421620.915665.4947110.6764676.04457859.4889D22962.50194694.96562240.28091777.2502417.3801990.87541302.90781189.0352D33901.46527511.530823463.7648234740.7023471.16711287.30197116.482267892.5484D454753.9636314083.215512238618.7632201244100.95713495.355929928.22602792922.933156051950.0538D582.4224146.5039582.2643583.835337.053966.1789367.3434756.0658D6103.4789156.2744569.0643558.900441.319173.8115335.2977747.2898D72545.60924065.91922422.9089921.0806362.5921851.43431237.22371099.3145D83486.90555656.67052660.4023973.9301453.95941086.77201448.3273922.6883D92728.54504209.19451468.15164644.7662439.10821067.40701219.62571188.9104D10126.8498204.54741926.103421620.915665.4947110.6764676.04457859.4889Table 7
EMAE withɛ∼N0,1 and p=4.
n2550ρ0.850.950.990.9990.850.950.990.999OLS1.43822.04046.420119.85980.91551.35424.540214.6991D11.06411.38224.723516.18450.76101.00813.178112.1143D21.03851.19611.27461.41090.76340.98071.19830.9127D31.04671.20901.45122.06720.76420.98261.28301.2601D41.12051.31412.16834.79160.77741.00731.59702.6147D50.98921.12981.08730.65540.75240.96171.13840.6004D60.99641.12921.04820.49180.75590.96571.12930.5412D71.02601.16071.06840.59860.76380.97571.13340.5802D81.03701.16651.16481.10600.76680.97641.15430.7462D91.06111.27613.918315.58640.77971.01662.508311.4210D101.06411.38224.723516.18450.76101.00813.178112.1143n75100OLS0.67470.96653.05779.88980.53130.76102.45577.8329D10.58210.76821.98537.79470.46970.62491.56875.9675D20.58630.76861.19980.90610.47320.62801.14160.9495D30.58680.76851.23441.09590.47360.62811.16001.0734D40.59470.78011.36011.80000.47880.63591.23041.5178D50.58000.75681.17770.77170.46910.62051.12670.8766D60.58170.75911.17370.74980.47030.62171.12410.8647D70.58770.76821.17950.76340.47450.62881.13160.8719D80.59030.77011.18070.82800.47650.63091.12990.9061D90.59490.78851.63466.96040.47780.64141.36345.1163D100.58210.76821.98537.79470.46970.62491.56875.9675Bold values indicate the minimum MAE.Table 8
EMAE withɛ∼N0,1 and p=8.
n2550ρ0.850.950.990.9990.850.950.990.999OLS4.19066.120519.750362.49921.75932.45567.725024.4799D13.00254.498416.110652.38291.50641.96796.115321.7759D22.39712.64312.54342.68821.52121.92152.74981.8304D32.46152.73783.10694.63261.52451.92442.84522.3196D42.98823.39616.490016.61391.57291.99233.46695.2519D52.13712.38512.16071.05171.48241.86542.68141.5286D62.11882.35392.11300.94231.48711.86952.67711.5090D72.26612.48842.15341.07391.51171.90562.70411.5362D82.41272.58292.33842.27301.53481.93022.70101.7221D92.37262.60124.03928.98551.53521.92252.78932.5095D103.00254.498416.110652.38291.50641.96796.115321.7759N75100OLS1.54762.20207.042122.27631.40111.97556.209119.5233D11.32141.75625.508119.65491.21961.61624.758117.2197D21.33691.71912.69361.84121.23251.60312.69681.9021D31.33731.72022.75412.19511.23001.60072.73652.1581D41.37021.76683.16704.33491.24591.62273.02163.7254D51.30431.67412.64901.66601.21001.57252.66631.7887D61.30681.67602.64361.65301.21171.57442.66401.7807D71.33201.71162.67241.67161.23041.60002.68801.7933D81.34331.72452.66381.77371.23471.60512.67951.8615D91.34821.72222.69312.18981.23491.60122.68462.0897D101.32141.75625.508119.65491.21961.61624.758117.2197Bold values indicate the minimum MAE.


## 3.3. Results and Discussion

The EMSE and EMAE values of the new and existing Liu estimators are presented in Tables1–6 and 7, 8 respectively. The performance of the LEs is evaluated by varying the values of factors such as multicollinearity, error variance, sample size, and the explanatory variables. A general remark from the literature is that these factors have the significant effect on the simulation design. The effect of each factor on EMSE and EMAE of estimators is discussed below:The first factor we considered is the effect of multicollinearity on the EMSE and EMAE of estimators. The EMSE and EMAE of all the estimators increase by increasing the degree of multicollinearity generally except D5–D8. Estimators D5–D8 first increase when multicollinearity increases from mild to high and decrease for severe multicollinearity. It is evident from these tables that the LE is always superior to the OLS estimator. For mild multicollinearity (ρ = 0.80), D5 outperforms others. Estimators D6 and D7 are the close competitors to D5. When the degree of multicollinearity is considered to be high (ρ = 0.99) or severe (ρ = 0.999), then the proposed estimators D6 and D7 outperform generally. However, the performance of D6 is good among others, because it yields lowest EMSE and EMAE when the degree of multicollinearity is high or severe.Secondly, the effect of sample size on the EMSE and EMAE of estimators is considered. In general, the EMSE and EMAE of OLS and LE decrease by raising the sample size. The proposed estimators D6 and D7 exhibit lowest EMSE in most of the cases, while D5 remains the closest competitor to the proposed estimators.In the third case, we varied the number of explanatory variables from 4 to 8. It is found that EMSE and EMAE of all the estimators increase. But the performance pattern remains the same as in the case of multicollinearity and sample size. It is seen that the increase in the EMSE of OLS estimator is relatively higher than all LEs. LE with Liu parameter D6 exhibits lowest EMSE and EMAE for both the cases of explanatory variables.Finally, the performance of estimators is compared by varying error variance. When the error variance increases from one to five, the EMSE of all the estimators deteriorates, except the new estimator D6, which is least affected.The concluded remarks from Tables1–8 are that the new estimators D6 and D7 have shown better performance than the other existing LEs in terms of smaller EMSE and EMAE, in situations of high multicollinearity disregarding the values of other factors. The proposed estimators also outperform the OLS estimator substantially. Therefore, based on EMSE and EMAE criterion, we can infer that the new estimator D6 is more efficient and is the best choice for the practitioners in the presence of high and severe multicollinearity.


## 4. Applications

Since the simulation evidence is not enough to judge the performance of the proposed estimators, because the study is usually conducted assuming some ideal conditions, in practice, the ideal conditions may not be met. Therefore, contrary to the previous section, we used two real applications taken from the books of [2, 15] in order to compare the performance of our proposed estimators in practical situations.


### 4.1. Portland Cement Data

The first numerical example used in this study is the Portland cement dataset taken from [15] to compare the performance of new estimators in applied scenario. Some authors named this data as Hald’s or chemical dataset and has been widely used in other literature; see [16–22]. The dataset consists of 13 observations and is given in Table 9. The response variable (y) is to study the heat evolved after 180 days of curing and measured in calories/gram of cement with 40% water at 350 C (950 F). Besides, the response variable (y), the four explanatory variables considered are the clinker compounds (CALCD) defined as(i)
X1: Tricalcium aluminate (3CaO∗Al2 O3)(ii)
X2: Tricalcium silicate (3CaO∗SiO2)(iii)
X3: Tetracalcium aluminoferrite (4CaO∗ Al2 O3∗ Fe2 O3)(iv)
X4: Dicalcium silicate (2CaO∗SiO2)Table 9
Portland cement data [15].
Observationx1x2x3x4y1129155274.321156820104.3372666078.5475263395.951155922109.26113184787.67131224472.58254182293.19371176102.7101166912113.3111068812109.412140233483.8132147426115.9The model is defined as(18)y=β0+β1X1+β2X2+β3X3+β4X4+ε.Condition number (CN) is used to measure the severity of multicollinearity among explanatory variables [23]. It can be defined as(19)CN=λmaxλmin,where λmax and λmin are the maximum and minimum eigenvalues of the matrix X′X, respectively. Following [2], a rule of thumb is that multicollinearity is moderate if the CN is between 10 and 30, high if it is between 30 and 100, and severe when it is greater than 100. The eigenvalues and CN for this dataset are computed and presented in Table 10. From Table 10, it is concluded that high multicollinearity exists among the explanatory variables. The Shapiro-Wilk (W) normality test is used to test the normality of response variable. We obtain the value test statistic W = 0.96967 and p-value = 0.8903, which shows that the response variable is normal at 5% level of significance. MSE of OLS and Liu estimators from [1] can be written as(20)MSEα^=σ2∑j=1p1λj,MSEα^d=σ2∑j=1pλj+d2λjλj+12+d−12∑j=1pα^j2λj+12.Table 10
Eigenvalues and condition number of Portland cement data.
λ1λ2λ3λ4CN2.2357 (55.89)1.57610.18660.001637.38064Parenthesis value gives the maximum percentage of first eigenvalue.The estimated values ford, coefficients, and MSE of estimators are presented in Table 11. This table shows that the LEs have smaller MSE than OLS. However, among the LEs, estimator D5 and new estimators D6 and D7 outperform others; therefore, they are highly efficient among others.Table 11
Estimated values ford, MSE and regression coefficients of Portland cement data.
EstimatorsdMSEβ^1β^2β^3β^4OLS—16.4515−0.65696−0.008310.302770.38774D10.460993.6085−0.64423−0.008080.252390.18274D20.026760.2726−0.63398−0.007900.211800.01759D30.128680.4937−0.63639−0.007950.221330.05635D40.460993.6085−0.64423−0.008080.252390.18274D50.000200.2716−0.63336−0.007890.209320.00749D60.000040.2716−0.63335−0.007890.209300.00743D70.005880.2698−0.63349−0.007890.209850.00965D80.075310.3349−0.63513−0.007920.216340.03605D90.244281.1617−0.63912−0.007990.232130.10032D100.460993.6085−0.64423−0.008080.252390.18274Bold values indicate the minimum MSE.


### 4.2. Prediction Interval

In this section, 95% mean prediction interval of all the estimators is computed from the Portland cement dataset. We consider the following values of explanatory variables:X0’ =  (X10, X20, X30, X40) = (23, 79, 27, 65).100 (1 −α) % mean prediction interval for the response variable is defined as(21)OLS:y^0±t1−α/2,vvary^0,LIU:y˜0±t1−α/2,vvary˜0,where y^0=β^1X10+β^2X20+β^3X30+β^4X40, y˜0=β˜1X10+β˜2X20+β˜3X30+β˜4X40, β^ and β˜ are the OLS and Liu estimators, respectively. t1−α/2 is the 1−α/2 quantile from the Student’s t-distribution with v=n−p degrees of freedom. vary^0=σ^2X0′X′X−1X0 and vary˜0=σ^2X0′QdX′X−1Qd′X0, Qd=X′X+I−1X′X+dI. For details, see [2, 4]. The results for the 95% mean prediction interval are given in Table 12. It can be seen from the table that the new estimator D6 gives the best mean prediction interval among all other estimators.Table 12
95% mean prediction interval for the Portland cement data.
EstimatorsLowerUpperDifferenceOLS265.5974300.8197566.4171D1130.2433136.7162266.9595D2−21.24854.560425.8089D3−46.803735.551682.3553D4130.2433136.7162266.9595D5−14.6542−3.450611.2037D6−14.6155−3.497711.1178D7−16.0524−1.749214.3033D8−33.412419.314052.7264D9−75.827270.7413146.5684D10130.2433136.7162266.9595Bold values indicate the best prediction interval.


### 4.3. US Crime Data

The US crime dataset is taken from the book of [2] to explain the crime rate in relation to the eight socioeconomic variables in 47 states in the United States for the year 1960. Response variable and eight explanatory variables are defined as follows:(i)
y = crime rate, number of offenses reported to police per million population(ii)
X1 = mean number of years of schooling times 10 for persons age 25 or older(iii)
X2 = 1960 per capita expenditure on police by state and local government(iv)
X3 = 1959 per capita expenditure on police by state and local government(v)
X4 = number of males per 1000 females(vi)
X5 = state population size in hundred thousands(vii)
X6 = unemployment rate of urban males per 1000 of age 35–39(viii)
X7 = median value of transferable goods and assets or family income in tens of dollars(ix)
X8 = the number of families per 1000 earnings 1/2 the median incomeTable13 gives the eigenvalues and condition number for these data, which shows that moderately high multicollinearity is present. The value of test statistic from Shapiro-Wilk normality test is W = 0.99379 with p-value = 0.9967, which shows that the response variable is normal at 5% level of significance. MSE results are tabulated in Table 14. From Table 14, we see that MSE of LEs outperforms OLS. Among all LEs, the proposed estimator D6 showed smallest MSE, which supports the simulation results given in Section 3.Table 13
Eigenvalues and condition number of US Crime data.
λ1λ2λ3λ4λ5λ6λ7λ8CN4.0851 (51.06)1.83900.97390.51190.35220.15360.07860.005726.77096Parenthesis value gives the maximum percentage of first eigenvalue.Table 14
MSE of US Crime data.
EstimatorsUS crime dataOLS72.5334D135.5803D23.2247D36.9318D459.4088D53.0783D63.0777D73.1067D83.1654D93.2047D1035.5803Bold values indicate the minimum MSE.Therefore, based on simulation results and illustrative examples, we recommend the use of LE with Liu parameter D6 and D7 to practitioners in presence of high or severe multicollinearity.


## 4.1. Portland Cement Data

The first numerical example used in this study is the Portland cement dataset taken from [15] to compare the performance of new estimators in applied scenario. Some authors named this data as Hald’s or chemical dataset and has been widely used in other literature; see [16–22]. The dataset consists of 13 observations and is given in Table 9. The response variable (y) is to study the heat evolved after 180 days of curing and measured in calories/gram of cement with 40% water at 350 C (950 F). Besides, the response variable (y), the four explanatory variables considered are the clinker compounds (CALCD) defined as(i)
X1: Tricalcium aluminate (3CaO∗Al2 O3)(ii)
X2: Tricalcium silicate (3CaO∗SiO2)(iii)
X3: Tetracalcium aluminoferrite (4CaO∗ Al2 O3∗ Fe2 O3)(iv)
X4: Dicalcium silicate (2CaO∗SiO2)Table 9
Portland cement data [15].
Observationx1x2x3x4y1129155274.321156820104.3372666078.5475263395.951155922109.26113184787.67131224472.58254182293.19371176102.7101166912113.3111068812109.412140233483.8132147426115.9The model is defined as(18)y=β0+β1X1+β2X2+β3X3+β4X4+ε.Condition number (CN) is used to measure the severity of multicollinearity among explanatory variables [23]. It can be defined as(19)CN=λmaxλmin,where λmax and λmin are the maximum and minimum eigenvalues of the matrix X′X, respectively. Following [2], a rule of thumb is that multicollinearity is moderate if the CN is between 10 and 30, high if it is between 30 and 100, and severe when it is greater than 100. The eigenvalues and CN for this dataset are computed and presented in Table 10. From Table 10, it is concluded that high multicollinearity exists among the explanatory variables. The Shapiro-Wilk (W) normality test is used to test the normality of response variable. We obtain the value test statistic W = 0.96967 and p-value = 0.8903, which shows that the response variable is normal at 5% level of significance. MSE of OLS and Liu estimators from [1] can be written as(20)MSEα^=σ2∑j=1p1λj,MSEα^d=σ2∑j=1pλj+d2λjλj+12+d−12∑j=1pα^j2λj+12.Table 10
Eigenvalues and condition number of Portland cement data.
λ1λ2λ3λ4CN2.2357 (55.89)1.57610.18660.001637.38064Parenthesis value gives the maximum percentage of first eigenvalue.The estimated values ford, coefficients, and MSE of estimators are presented in Table 11. This table shows that the LEs have smaller MSE than OLS. However, among the LEs, estimator D5 and new estimators D6 and D7 outperform others; therefore, they are highly efficient among others.Table 11
Estimated values ford, MSE and regression coefficients of Portland cement data.
EstimatorsdMSEβ^1β^2β^3β^4OLS—16.4515−0.65696−0.008310.302770.38774D10.460993.6085−0.64423−0.008080.252390.18274D20.026760.2726−0.63398−0.007900.211800.01759D30.128680.4937−0.63639−0.007950.221330.05635D40.460993.6085−0.64423−0.008080.252390.18274D50.000200.2716−0.63336−0.007890.209320.00749D60.000040.2716−0.63335−0.007890.209300.00743D70.005880.2698−0.63349−0.007890.209850.00965D80.075310.3349−0.63513−0.007920.216340.03605D90.244281.1617−0.63912−0.007990.232130.10032D100.460993.6085−0.64423−0.008080.252390.18274Bold values indicate the minimum MSE.


## 4.2. Prediction Interval

In this section, 95% mean prediction interval of all the estimators is computed from the Portland cement dataset. We consider the following values of explanatory variables:X0’ =  (X10, X20, X30, X40) = (23, 79, 27, 65).100 (1 −α) % mean prediction interval for the response variable is defined as(21)OLS:y^0±t1−α/2,vvary^0,LIU:y˜0±t1−α/2,vvary˜0,where y^0=β^1X10+β^2X20+β^3X30+β^4X40, y˜0=β˜1X10+β˜2X20+β˜3X30+β˜4X40, β^ and β˜ are the OLS and Liu estimators, respectively. t1−α/2 is the 1−α/2 quantile from the Student’s t-distribution with v=n−p degrees of freedom. vary^0=σ^2X0′X′X−1X0 and vary˜0=σ^2X0′QdX′X−1Qd′X0, Qd=X′X+I−1X′X+dI. For details, see [2, 4]. The results for the 95% mean prediction interval are given in Table 12. It can be seen from the table that the new estimator D6 gives the best mean prediction interval among all other estimators.Table 12
95% mean prediction interval for the Portland cement data.
EstimatorsLowerUpperDifferenceOLS265.5974300.8197566.4171D1130.2433136.7162266.9595D2−21.24854.560425.8089D3−46.803735.551682.3553D4130.2433136.7162266.9595D5−14.6542−3.450611.2037D6−14.6155−3.497711.1178D7−16.0524−1.749214.3033D8−33.412419.314052.7264D9−75.827270.7413146.5684D10130.2433136.7162266.9595Bold values indicate the best prediction interval.


## 4.3. US Crime Data

The US crime dataset is taken from the book of [2] to explain the crime rate in relation to the eight socioeconomic variables in 47 states in the United States for the year 1960. Response variable and eight explanatory variables are defined as follows:(i)
y = crime rate, number of offenses reported to police per million population(ii)
X1 = mean number of years of schooling times 10 for persons age 25 or older(iii)
X2 = 1960 per capita expenditure on police by state and local government(iv)
X3 = 1959 per capita expenditure on police by state and local government(v)
X4 = number of males per 1000 females(vi)
X5 = state population size in hundred thousands(vii)
X6 = unemployment rate of urban males per 1000 of age 35–39(viii)
X7 = median value of transferable goods and assets or family income in tens of dollars(ix)
X8 = the number of families per 1000 earnings 1/2 the median incomeTable13 gives the eigenvalues and condition number for these data, which shows that moderately high multicollinearity is present. The value of test statistic from Shapiro-Wilk normality test is W = 0.99379 with p-value = 0.9967, which shows that the response variable is normal at 5% level of significance. MSE results are tabulated in Table 14. From Table 14, we see that MSE of LEs outperforms OLS. Among all LEs, the proposed estimator D6 showed smallest MSE, which supports the simulation results given in Section 3.Table 13
Eigenvalues and condition number of US Crime data.
λ1λ2λ3λ4λ5λ6λ7λ8CN4.0851 (51.06)1.83900.97390.51190.35220.15360.07860.005726.77096Parenthesis value gives the maximum percentage of first eigenvalue.Table 14
MSE of US Crime data.
EstimatorsUS crime dataOLS72.5334D135.5803D23.2247D36.9318D459.4088D53.0783D63.0777D73.1067D83.1654D93.2047D1035.5803Bold values indicate the minimum MSE.Therefore, based on simulation results and illustrative examples, we recommend the use of LE with Liu parameter D6 and D7 to practitioners in presence of high or severe multicollinearity.


## 5. Concluding Remarks

In this paper, we introduced a new quantile based method to estimate the Liu parameter in order to minimize the variance and circumvent the problem of multicollinearity. Extensive Monte Carlo simulations were carried out to evaluate the performance of estimators with MSE and MAE criterions by varying the values of different factors such as sample sizes, number of explanatory variables, error variances, and multicollinearity. Results from the simulation study revealed that the LE performed generally better than the traditional OLS estimator.The LE is a robust choice than the OLS when the problem of multicollinearity is present. Also, the proposed estimator D6 performs better than other considered estimators in many evaluated instances particularly when the problem of multicollinearity is extremely high. Estimator D5 is the closest competitor to D6. Furthermore, the benefits of the new estimators are evidently confirmed in the two empirical applications. Based on the simulation results along with the real applications, we conclude that the Liu method with proposed estimator D6 is the best choice for practitioners to overcome the problem of multicollinearity.



---

*Source: 1772328-2021-07-10.xml*