# Modified Least Trimmed Quantile Regression to Overcome Effects of Leverage Points

Authors: Habshah Midi; Taha Alshaybawee; Mohammed Alguraibawi
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

Quantile regression estimates are robust for outliers iny direction but are sensitive to leverage points. The least trimmed quantile regression (LTQReg) method is put forward to overcome the effect of leverage points. The LTQReg method trims higher residuals based on trimming percentage specified by the data. However, leverage points do not always produce high residuals, and hence, the trimming percentage should be specified based on the ratio of contamination, not determined by a researcher. In this paper, we propose a modified least trimmed quantile regression method based on reweighted least trimmed squares. Robust Mahalanobis’ distance and GM6 weights based on Gervini and Yohai’s (2003) cutoff points are employed to determine the trimming percentage and to detect leverage points. A simulation study and real data are considered to investigate the performance of our proposed methods.


---

## Body

## 1. Introduction

Quantile regression (QReg) has received much attention since the seminal work of Koenker and Bassett [1]. It can be considered as one of the important statistical breakthroughs in recent decades. The desirable advantages of quantile regression led to its application in wide areas of sciences such as in medicine, financial, economics, agriculture, environment, and others [2, 3]. QReg is an extension of the mean regression model to conditional of the different quantiles of the response variable distribution. Therefore, QReg is able to provide much more detailed stochastic relationship among random variables.Consider the following regression model:(1)yi=xi′β+εi,where yi is an n×1 vector of a response variable, xi is a k×1 vector of covariates variables, β is a vector of unknown parameters, and εi is an n×1 vector of error terms. For any τ-quantiles in the interval (0, 1), the parameter βτ can be estimated consistently as the solution to the following optimization problem:(2)minβ∑i=1nρτyi−xi′βτ,where ρτ. is the check function, defined as(3)ρτu=uτ−Iu<0,where I. denotes the indicator function.One of the important advantages of quantile regression is the insensitivity for outliers and heavy tailed distribution for error term. This robustness of QReg for outliers arises because of the nature of the check function which is shown in (3) (see [3–5]). Similar to M-estimator regression, QReg is not robust when the predictor variables contain outliers which are called high leverage points (HLPs) [6]. There are some attempts to overcome the effect of HLPs and maximize the breakdown point of QReg. Giloni et al. [7] proposed a weighting method to increase the breakdown point and cope with HLP, based on the blocked adaptive computationally efficient outlier nominators (BACON) method that is proposed by Billor et al. (2000), in which a clean subset is chosen via their algorithm. The limitation of the weighting method is that it can be used with small numbers of regressers (often, one or two regresser variables). Rousseeuw and Hubert [8] proposed the regression depth as an extended version for regression quantile. They pointed out that the depth quantiles is robust to HLPs. Adrover et al. [9] presented a robust estimation method that is unaffected by leverage points and, at the same time, maximizes the breakdown point. The disadvantages of the weighting method and depth quantiles are computational complexity and nonstandard asymptotic distributions Neykov et al. [10].Recently, least trimmed QReg is proposed by Neykov et al. [10] to reduce the effects of HLPs. This method is a generalization of the location estimator that was proposed by Tableman [11] and least trimmed absolute deviation proposed by Hawkins and Olive [12]. Neykov et al. [10] proved the consistency of the least trimmed quantile regression method and discussed the breakdown point of the estimators. The limitation of this method is that the trimming percentage is a constant whereby the trimmed data may be lower or higher than the actual contamination percentage of the data. The least trimmed quantile method minimizes the quantile residuals in (2) for the subset (h) out of the sample size (n). However, it is important to mention that the leverage point is not affected by residuals. Therefore, this method does not correctly detect the high leverage points.In this paper, we proposed a new algorithm to develop the least trimmed quantile regression method and to overcome these disadvantages in the existing methods. The new proposed algorithm integrates the reweighted least trimmed method that proposed by Čížek [13] with QReg to determine the trimming percentage and robust Mahalanobis’ distance to identify the HLPs. In addition, we employ the Gervini and Yohai [14] technique to compute the cutoff point and new weights for the QReg. Besides that, RMD is used to detect the leverage points.


## 2. Least Trimmed Quantile Regression (LTQReg)

Least trimmed squares (LTS) method is a robust estimation technique proposed by Rousseeuw [15] by minimizing the following objective function:(4)minβ∑i=1hεi2β,where εi2β is the i-th order statistic squares residuals, ε12β, ε22β, …, εn2β, and h is a subset out of (n). The trimming constant h=n1−α+1, where α is a ratio of trimming. When h=n+p+1/2, the highest breakdown point of the LTS estimator will be achieved (0.50) [16]. Roozbeh and Hamzah [17] developed the LTS method for restricted semiparametric regression models. Based on the least trimmed squares (LTS) method, robust ridge and nonridge type estimators were developed by Roozbeh [18] in semiparametric model regression when the errors are dependent. Roozbeh et al. [19] introduced some alternative robust estimators based on a penalization scheme, whereas a nonlinear integer programming was used.Neykov et al. [10] proposed the least trimmed quantile regression (LTQReg) as an efficient and robust method to overcome the effect of HLPs on QReg. LTQReg is defined as follows:(5)β^τ=argminβ∑i=1hρτεiβ,where ρτεiβ is defined as in (2) and (3). Neykov et al. [10] proved that when the trimming constant h=n+NX+1/2, the breakdown point of LTQReg estimator is asymptotically equal to 0.50, where NX is the maximum number of explanatory variables. Müller [20] and Neykov et al. [10] pointed out that NX=p−1.The LTQReg method is based on the smallest quantile errors to reduce the influence of leverage points. In this situation, we would like to ask the following question: is the error values of the QReg and LS will be high for all leverage points? We most answer to this question by the following example. Let us consider the simple linear regression modelyi=β0+β1xi+εi, with a sample size of n = 50, and the independent variable xi is uniformly distributed −1,1. Let β0=β1=1 and εi∼N0,1.In order to illustrate the effect of leverage points and outliers on the error term, 20% of the observations are contaminated by replacing the first 10 observations with contaminated observations. We consider three cases of contaminations: outliers, HLPs, and both outliers and HLPs simultaneously. The first 10 observations of the explanatory variable and dependent variable are contaminated as follows:xi∼U−50,50 and yi∼U50,100. Least squares (LS) and QReg at three quantiles (0.25, 0.50, and 0.75) were then applied to the data. In this example, we want to investigate if the LS and QReg produced high errors in all contamination scenarios which are suitable for LTQReg.For all the three cases of contaminations, the fitted residuals are plotted as shown in Figures1–3.Figure 1
The fitted residuals for LS and QReg at three quantiles (0.25, 0.50, and 0.75) for contaminated data with outliers.Figure 2
The fitted residuals for LS and QReg at three quantiles (0.25, 0.50, and 0.75) for contaminated data with HLPs.Figure 3
The fitted residuals for LS and QReg at three quantiles (0.25, 0.50, and 0.75) for contaminated data with both outliers and HLPs.Figures1–3 clarify influences of outliers, HLPs, and both on LS and QReg in different quantiles. In Figures 1 and 3, we can see clearly that when the data are contaminated by outliers, the first 10 observations have highest residuals for both LS and QReg in different quantiles. On the contrary, in Figure 2, when the data are contaminated by HLPs, the residuals of LS and QReg are not affected by HLPs. From Figures 1–3, we can conclude that the outlier observations have a direct effect on the residuals, whereas the leverage points have no effect on the residuals. Hence, we can say that the LTQReg method is not an effective method to reduce the effect of leverage points because it is based on trimming the highest (n − h) residuals.


## 3. Modified Least Trimmed Quantile Regression (MLTQReg)

In this section, we will discuss the modified LTQReg method to determine the rate of contamination data and the best trimming percentage. Three modified methods will be discussed in this section based on the reweighted least trimmed squares (RWLTS) method which was proposed by Čížk [13] depending on hard rejection weights [16] and combined to the LTQReg method to robustify the weighted least squares method. The hard rejection weights in the RWLTS method are defined as(6)wi=1,ifui<tn,0,ifui≥tn,where ui′s are the standardized of regression residuals and tn>0 is the cutoff point that was adapted by Gervini and Yohai [14]. The cutoff point value is computed by comparing the empirical distribution function Gn+ of standardized absolute residuals with the distribution function G0+ of absolute residuals under the assumed model. The friction of unusual observation in the sample dn can be measured as(7)dn=sup︸t≥kGn+t−G0+t+,where k = 2.5 [16]. Therefore, the tn value is set to the 1−dnth quantile of Gn+t as follows:(8)tn=mint:Gn+t≥1−dn.The procedure of the reweighted least trimmed square method [13] can be described in two steps. The first step is determining the trimming constant h based on the weights that are given in (6), defined as(9)h=∑i=1nwi.The second step is applying the LTS method depending on the trimming constanth that is computed in the first step.To increase breakdown points of the proposed method, a high breakdown estimator LTS, LMS, or S are used as initial (see [13, 14]) and the robust weights are used to improve the efficiency.Next, we will describe three algorithms based on the RWLTS to improve the LTQReg [10].


### 3.1. Reweighted Least Trimmed Quantile Regression (RWLTQReg)

In this method, we combine the RWLTS with LTQReg to determine the trimming constant, and the algorithm for this method can be describe as follows:Step 1. Consider the LTQReg estimator as an initial estimate with high breakdown point and compute the standardized residualui for i=1,…,n.Step 2. Calculate hard rejection weights for the standardized residuals as(10)wiRes=Iui<tn,wheretn is the cutoff point of Gervini and Yohai [14] that is shown in (8).Step 3. Calculate the trimming constanthn based on the weights in equation (10), from the formula hn=∑i=1nwiRes.Step 4. Applying the LTQReg based on the algorithm that proposed by Neykov et al. [10] for the subset of the size hn, this procedure can be described as follows:(i)
Setr=0, select a subset with the size hn from the sample.(ii)
For the subsethn, use the QReg to estimate the coefficients β^τr.(iii)
For all observations in the sample, compute the residuals and then order the residuals asu1β^τr≤u2β^τr,…,≤unβ^τr, i=1,…,n.(iv)
Then, setr=r+1 and the new subset is considered the first hn.(v)
For the new subset, Steps (ii), (iii), and (iv) were repeated. This procedure is repeated until convergence.


### 3.2. Modified Least Trimmed Quantile Regression Based on RMD (RMD-LTQReg)

In this algorithm, we used the modified least trimmed quantile regression (MLTQReg) and RMD to detect the leverage points. The algorithm is presented as follows:Step 1. Compute the RMD as follows:(11)RMDxi=xi−TX′CX−1xi−TX,whereT (X) and C (X) are the location and the shape estimates of MVE.Step 2. ComputewiRMD=IRMDxi<K, where K is the cutoff point computed as follows:(12)K=medianRMDxi+c∗madRMDxi,wheremadyi=medyi−medyj and c is a constant, 2 or 3.Step 3. As in Step 1 and 2 of the RWLTQReg algorithm, we computewiRes weights.Step 4. Find the final weights by combiningwiRMD with wiRes as follows:(13)wiFin=wiRMD,ifwiRMD=wiRes,0,ifwiRMD≠wiRes.Step 5. Hence, the trimming constant ...Step 6. We will apply Step 4 in the RWLTQReg algorithm for the subset of the sizehn from the sample and set probability zero for the leverage points to ensure that we will not start with the bad subset (contains leverage points), Rousseeuw and Van Driessen [21], which means the condition of wiRMD≠0 is satisfied (clean of leverage points).


### 3.3. Modified Least Trimmed Quantile Regression Based on GM6 Method (GM6-LTQReg)

The GM-estimator is proposed by Schweppe (see [22]) to reduce the influence of leverage points. Adrover et al. [9] showed that the breakdown of the GM-estimator was never higher than 1/p+1. Coakley and Hettmansperger [23] proposed the GM6-estimator to increase the breakdown point of the GM-estimator by using the least trimmed squares (LTS) as initial and RMD based on MVE to downweight leverage points. In this paper, we suggest using the GM6 weights to modify the LTQReg, and the procedure of this modification can be determined by following algorithm:Step 1. Fori=1,…,n, compute an initial estimate for the coefficients and the corresponding residuals εi, the high breakdown estimators (LTQReg), Neykov et al. [10].Step 2. Compute the scale of the residuals (se), as follows:(14)se=1.4826median of largestn−pof theεi.Step 3. Determine the standardized residualsui=εi/wi0×se, where wi0 is an initial weight computed as follows:(15)wi0=min1,χ0.95,p+12RMD2.Step 4. Hard rejection weights for the standardized residuals can be computed as follows:(16)wi=Iui<tn,wheretn is a Gervini and Yohai [14] cutoff point that is shown in (8).Step 5. Hence, the trimming parameter will be computed ashn=∑i=1nwi.Step 6. Step 4 in the RWLTQReg algorithm for the subset of the sizehn.


## 3.1. Reweighted Least Trimmed Quantile Regression (RWLTQReg)

In this method, we combine the RWLTS with LTQReg to determine the trimming constant, and the algorithm for this method can be describe as follows:Step 1. Consider the LTQReg estimator as an initial estimate with high breakdown point and compute the standardized residualui for i=1,…,n.Step 2. Calculate hard rejection weights for the standardized residuals as(10)wiRes=Iui<tn,wheretn is the cutoff point of Gervini and Yohai [14] that is shown in (8).Step 3. Calculate the trimming constanthn based on the weights in equation (10), from the formula hn=∑i=1nwiRes.Step 4. Applying the LTQReg based on the algorithm that proposed by Neykov et al. [10] for the subset of the size hn, this procedure can be described as follows:(i)
Setr=0, select a subset with the size hn from the sample.(ii)
For the subsethn, use the QReg to estimate the coefficients β^τr.(iii)
For all observations in the sample, compute the residuals and then order the residuals asu1β^τr≤u2β^τr,…,≤unβ^τr, i=1,…,n.(iv)
Then, setr=r+1 and the new subset is considered the first hn.(v)
For the new subset, Steps (ii), (iii), and (iv) were repeated. This procedure is repeated until convergence.


## 3.2. Modified Least Trimmed Quantile Regression Based on RMD (RMD-LTQReg)

In this algorithm, we used the modified least trimmed quantile regression (MLTQReg) and RMD to detect the leverage points. The algorithm is presented as follows:Step 1. Compute the RMD as follows:(11)RMDxi=xi−TX′CX−1xi−TX,whereT (X) and C (X) are the location and the shape estimates of MVE.Step 2. ComputewiRMD=IRMDxi<K, where K is the cutoff point computed as follows:(12)K=medianRMDxi+c∗madRMDxi,wheremadyi=medyi−medyj and c is a constant, 2 or 3.Step 3. As in Step 1 and 2 of the RWLTQReg algorithm, we computewiRes weights.Step 4. Find the final weights by combiningwiRMD with wiRes as follows:(13)wiFin=wiRMD,ifwiRMD=wiRes,0,ifwiRMD≠wiRes.Step 5. Hence, the trimming constant ...Step 6. We will apply Step 4 in the RWLTQReg algorithm for the subset of the sizehn from the sample and set probability zero for the leverage points to ensure that we will not start with the bad subset (contains leverage points), Rousseeuw and Van Driessen [21], which means the condition of wiRMD≠0 is satisfied (clean of leverage points).


## 3.3. Modified Least Trimmed Quantile Regression Based on GM6 Method (GM6-LTQReg)

The GM-estimator is proposed by Schweppe (see [22]) to reduce the influence of leverage points. Adrover et al. [9] showed that the breakdown of the GM-estimator was never higher than 1/p+1. Coakley and Hettmansperger [23] proposed the GM6-estimator to increase the breakdown point of the GM-estimator by using the least trimmed squares (LTS) as initial and RMD based on MVE to downweight leverage points. In this paper, we suggest using the GM6 weights to modify the LTQReg, and the procedure of this modification can be determined by following algorithm:Step 1. Fori=1,…,n, compute an initial estimate for the coefficients and the corresponding residuals εi, the high breakdown estimators (LTQReg), Neykov et al. [10].Step 2. Compute the scale of the residuals (se), as follows:(14)se=1.4826median of largestn−pof theεi.Step 3. Determine the standardized residualsui=εi/wi0×se, where wi0 is an initial weight computed as follows:(15)wi0=min1,χ0.95,p+12RMD2.Step 4. Hard rejection weights for the standardized residuals can be computed as follows:(16)wi=Iui<tn,wheretn is a Gervini and Yohai [14] cutoff point that is shown in (8).Step 5. Hence, the trimming parameter will be computed ashn=∑i=1nwi.Step 6. Step 4 in the RWLTQReg algorithm for the subset of the sizehn.


## 4. Simulation Study

In this section, the Monte Carlo simulation study is presented to compare the performances of some existing methods such as LTQReg [10] and QR [1] with our proposed methods RWLTQReg, RMD-LTQReg, and GM6-LTQReg.Following Neykov et al. [10], two explanatory variables (xi1 and xi2) are generated with large sample size (n = 100 and 200) from the following classical heteroscedastic multiple linear regression model:(17)yi=b0+b1xi1+b2xi2+σiεi,σi=exp0.11xi1+xi21/2,where we assume that the coefficients b0=b1=b2=1, and the error term εi is distributed as N0,1. Also, two experiments are considered with different distribution for explanatory and response variables with three levels of contamination δ=10%,20%,and30%. The trimming percentage for the LTQReg will be considered as (0.20, 0.30).


### 4.1. The First Experiment

In this experiment, a distribution of the variables in the model is the uniform distribution (Unif) with parameters−10,10. The variables are contaminated with different percentages, where the explanatory variables are contaminated as xij∼Unif−30,−20, j = 1, 2, and the response variable is contaminated as yi∼Unif40,80.


### 4.2. The Second Experiment

In this experiment, a distribution of explanatory variables is set as normal distributionN0,1. The variables are contaminated with different percentages, where the explanatory variables are contaminated as xij∼N−10,3, j = 1, 2, and the response variable is contaminated as yi∼N20,3.The contamination is done by replacement of clean data by outlying data in both explanatory and response variables. Letm=δ×n, and the explanatory variables are contaminated by replacing i=intgerm/3,…,m clean observations with outlying observations, whereas the response variable yi is contaminated by replacing i=1,…,intger2m/3 clean observations with outlying observations.Figures4 and 5 show the spread shape of generating data. It can be seen clearly that these data contain leverage points (outlying in the x direction) and outliers (outlying in the y direction) and influence observations (outlying in both x and y directions in the same time).Figure 4
The spread shape of the generated data in the first experiment at the three levels of contamination (0.10, 0.20, and 0.30) when the sample sizen = 100 and 200.Figure 5
The spread shape for generated data in the second experiment at the three levels of contamination (0.10, 0.20, and 0.30) when the sample sizen = 100 and 200.At the three quantiles 0.25, 0.50, and 0.75, the generated data in different contamination percentages (10%, 20%, and 30%) are fitting via the proposed methods (RWLTQReg, RMD-LTQReg, and GM6-LTQReg) and the existing methods (QReg and LTQReg). Root of mean squares errors (RMSE=1/n∑i=1nyi−y^i2) and mean absolute errors (MAE=1/n∑i=1nyi−y^i) for the model and standard error for the parameters (SE=1/100∑r=1100β^jr−βjtrue2) are computed to evaluate our proposed methods.In Tables1 and 2, we reported the RMSE and MAE values for the first and second experiments. In these tables, we can see that RMSE and MAE values for all the methods at three quantiles are shown in the rows and three levels of contamination are shown in the columns. LTQReg (20%) and LTQReg (30%) show the least trimmed quantile method with 20% and 30% trim, respectively. The results in these tables are the average of 100 replications for the two experiments of the simulation study.Table 1
RMSE and MAE values for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg at the three quantiles 0.25, 0.50, and 0.75, in the different contamination levels 10%, 20%, and 30% for the first experiment.
Sample sizeContamination rate10%20%30%TauMethodsRMSEMAERMSEMAERMSEMAE1000.25QReg23.76527.988634.525015.639529.603718.6166LTQReg (20%)1.01110.79971.35670.987017.52087.0508LTQReg (30%)0.82560.66680.96910.77081.28640.9324RWLTQReg0.56450.46220.63520.51714.52393.7996RMD-LTQReg0.61830.50490.65620.53100.67840.5492GM6-LTQReg0.94340.75431.03650.81771.24920.90290.50QReg23.24667.760322.878113.764726.821817.0167LTQReg (20%)0.77880.61141.15490.82528.34526.7991LTQReg (30%)0.59730.48030.74070.58396.78345.5281RWLTQReg0.39190.31571.31741.09604.01463.2788RMD-LTQReg0.43890.35440.46410.37390.48570.3917GM6-LTQReg0.69260.55120.90850.67447.69866.11820.75QReg17.198311.035323.778719.385928.488624.1319LTQReg (20%)4.06783.353814.982412.449122.111618.6020LTQReg (30%)3.61182.975113.130110.930419.602216.4446RWLTQReg4.10623.310910.13118.135615.685912.7729RMD-LTQReg0.62670.50880.68540.55600.77100.6227GM6-LTQReg2.97531.288116.117713.163424.413020.51512000.25QReg24.11818.125232.185914.420831.017018.4829LTQReg (20%)1.06040.85391.46441.084715.32586.3742LTQReg (30%)0.88400.72761.04970.84651.46601.0884RWLTQReg0.62560.52400.72070.59940.83080.6854RMD-LTQReg0.64790.53960.73010.60300.81010.6651GM6-LTQReg1.03880.83991.13690.90691.24980.98080.50QReg23.54867.933223.546812.854326.487816.6496LTQReg (20%)0.84200.67351.23090.89517.90885.8414LTQReg (30%)0.65620.53650.80530.64481.24680.8983RWLTQReg0.40060.33220.79930.66581.14820.9557RMD-LTQReg0.47480.39190.52440.43020.56960.4662GM6-LTQReg0.76380.61760.88880.70171.22700.81420.75QReg17.251911.033624.135919.372528.269223.0904LTQReg (20%)1.06530.856412.705710.625919.988716.5240LTQReg (30%)0.88390.725810.95699.150217.418414.3180RWLTQReg2.74742.26298.47836.888513.616611.0773RMD-LTQReg0.66510.55250.74080.61220.82330.6760GM6-LTQReg1.35061.027913.563411.305521.486017.7558Table 2
RMSE and MAE values for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg at the three quantiles 0.25, 0.50, and 0.75, in the different contamination levels 20%, 30%, and 40% for the second experiment.
Sample sizeContamination rate10%20%30%TauMethodsRMSEMAERMSEMAERMSEMAE1000.25QReg7.600722.7113712.639555.4064915.295707.76377LTQReg (20%)0.964390.795511.129110.908032.575471.52490LTQReg (30%)0.837760.695870.940680.771941.105280.89317RWLTQReg0.612780.507900.703930.580180.761340.62801RMD-LTQReg0.614120.504720.635760.519190.608210.49545GM6-LTQReg1.004620.826151.046730.852081.076110.873030.50QReg7.034032.498619.505624.9003611.695556.27158LTQReg (20%)0.756410.615551.346681.081312.437781.87786LTQReg (30%)0.624990.511601.044470.854311.674201.36147RWLTQReg0.394080.320700.688700.562721.013280.82816RMD-LTQReg0.441340.357570.450010.363100.451310.36386GM6-LTQReg0.751620.612541.105780.902011.496761.223310.75QReg6.482702.9863810.870285.6874613.196356.99549LTQReg (20%)1.277771.046502.764382.252733.105302.52145LTQReg (30%)1.191030.984392.329181.911002.541142.09051RWLTQReg0.939360.778601.596551.313191.739521.43453RMD-LTQReg0.625200.514220.645710.529440.642470.52357GM6-LTQReg1.335481.088092.512962.059252.424491.996872000.25QReg8.096252.9380212.500055.3967115.543027.89287LTQReg (20%)0.976600.814001.121260.913532.589691.53480LTQReg (30%)0.855190.719150.936520.780481.108760.90673RWLTQReg0.658750.556840.696500.587540.769610.64687RMD-LTQReg0.621500.522970.618010.519200.622470.52132GM6-LTQReg1.037880.858811.052230.866221.080090.886720.50QReg7.467372.695869.446304.8971211.811016.27112LTQReg (20%)0.760730.627291.145120.925002.459821.97166LTQReg (30%)0.626270.521520.927520.765561.817341.48822RWLTQReg0.413740.344920.686050.571051.070190.89003RMD-LTQReg0.418870.347170.436970.360760.417050.34530GM6-LTQReg0.771880.636161.013860.829941.739161.427380.75QReg7.450193.9761410.920265.6266913.216337.00179LTQReg (20%)1.618941.346172.748052.276453.176202.61515LTQReg (30%)1.419701.195612.362341.984662.655882.22840RWLTQReg1.099510.933511.742171.480321.947451.65112RMD-LTQReg0.615260.518050.634960.533800.641020.53548GM6-LTQReg1.784271.466442.573932.148422.577852.16814Table1 and 2 show that the RMSE and MAE values for the QReg method at the different quantiles and different levels of contamination are the highest. That is, this method is more affected than the other methods by outlying data that fall in both x and y directions. On the contrary, we can see that the proposed method RMD-LTQReg has the lowest values of RMSE and MAE in most cases. This indicates that the performance of RMD-LTQReg is better than the others. On the other hand, the RWLTQReg has better performance than the other methods except the RMD-LTQReg. In addition, we can see when the contamination levels are 20% and 30%, the GM6-LTQReg performance is better than LTQReg (20%) in most cases and LTQReg (30%) in the 30% contamination level.In Figures6–9, we can see that the SE values for the parameters that estimated by the RMD-LTQReg method are the smallest in almost all cases, which indicates the performance of the RMD-LTQReg method is the best among all studied methods. Also, these figures show clearly that the QReg method has high SE leads to worse performance. In addition, the rest of the methods used showed close results in most cases and were varying in some other cases. Therefore, it could be argued that it is difficult to determine which one is better than others.Figure 6
SE values for the estimated parameters at each quantile and contamination level for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg, and M6 = GM6-LTQReg) in the first experiment whenn = 100.Figure 7
SE values for the estimate parameters at each quantile and contamination level for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg, and M6 = GM6-LTQReg) in the first experiment whenn = 200.Figure 8
The SE values for the estimated parameters at each quantile and contamination level for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg, and M6 = GM6-LTQReg) in the second experiment whenn = 100.Figure 9
The SE values for the estimated parameters at each quantile and contamination levels for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg and M6 = GM6-LTQReg) in the second experiment whenn = 200.


## 4.1. The First Experiment

In this experiment, a distribution of the variables in the model is the uniform distribution (Unif) with parameters−10,10. The variables are contaminated with different percentages, where the explanatory variables are contaminated as xij∼Unif−30,−20, j = 1, 2, and the response variable is contaminated as yi∼Unif40,80.


## 4.2. The Second Experiment

In this experiment, a distribution of explanatory variables is set as normal distributionN0,1. The variables are contaminated with different percentages, where the explanatory variables are contaminated as xij∼N−10,3, j = 1, 2, and the response variable is contaminated as yi∼N20,3.The contamination is done by replacement of clean data by outlying data in both explanatory and response variables. Letm=δ×n, and the explanatory variables are contaminated by replacing i=intgerm/3,…,m clean observations with outlying observations, whereas the response variable yi is contaminated by replacing i=1,…,intger2m/3 clean observations with outlying observations.Figures4 and 5 show the spread shape of generating data. It can be seen clearly that these data contain leverage points (outlying in the x direction) and outliers (outlying in the y direction) and influence observations (outlying in both x and y directions in the same time).Figure 4
The spread shape of the generated data in the first experiment at the three levels of contamination (0.10, 0.20, and 0.30) when the sample sizen = 100 and 200.Figure 5
The spread shape for generated data in the second experiment at the three levels of contamination (0.10, 0.20, and 0.30) when the sample sizen = 100 and 200.At the three quantiles 0.25, 0.50, and 0.75, the generated data in different contamination percentages (10%, 20%, and 30%) are fitting via the proposed methods (RWLTQReg, RMD-LTQReg, and GM6-LTQReg) and the existing methods (QReg and LTQReg). Root of mean squares errors (RMSE=1/n∑i=1nyi−y^i2) and mean absolute errors (MAE=1/n∑i=1nyi−y^i) for the model and standard error for the parameters (SE=1/100∑r=1100β^jr−βjtrue2) are computed to evaluate our proposed methods.In Tables1 and 2, we reported the RMSE and MAE values for the first and second experiments. In these tables, we can see that RMSE and MAE values for all the methods at three quantiles are shown in the rows and three levels of contamination are shown in the columns. LTQReg (20%) and LTQReg (30%) show the least trimmed quantile method with 20% and 30% trim, respectively. The results in these tables are the average of 100 replications for the two experiments of the simulation study.Table 1
RMSE and MAE values for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg at the three quantiles 0.25, 0.50, and 0.75, in the different contamination levels 10%, 20%, and 30% for the first experiment.
Sample sizeContamination rate10%20%30%TauMethodsRMSEMAERMSEMAERMSEMAE1000.25QReg23.76527.988634.525015.639529.603718.6166LTQReg (20%)1.01110.79971.35670.987017.52087.0508LTQReg (30%)0.82560.66680.96910.77081.28640.9324RWLTQReg0.56450.46220.63520.51714.52393.7996RMD-LTQReg0.61830.50490.65620.53100.67840.5492GM6-LTQReg0.94340.75431.03650.81771.24920.90290.50QReg23.24667.760322.878113.764726.821817.0167LTQReg (20%)0.77880.61141.15490.82528.34526.7991LTQReg (30%)0.59730.48030.74070.58396.78345.5281RWLTQReg0.39190.31571.31741.09604.01463.2788RMD-LTQReg0.43890.35440.46410.37390.48570.3917GM6-LTQReg0.69260.55120.90850.67447.69866.11820.75QReg17.198311.035323.778719.385928.488624.1319LTQReg (20%)4.06783.353814.982412.449122.111618.6020LTQReg (30%)3.61182.975113.130110.930419.602216.4446RWLTQReg4.10623.310910.13118.135615.685912.7729RMD-LTQReg0.62670.50880.68540.55600.77100.6227GM6-LTQReg2.97531.288116.117713.163424.413020.51512000.25QReg24.11818.125232.185914.420831.017018.4829LTQReg (20%)1.06040.85391.46441.084715.32586.3742LTQReg (30%)0.88400.72761.04970.84651.46601.0884RWLTQReg0.62560.52400.72070.59940.83080.6854RMD-LTQReg0.64790.53960.73010.60300.81010.6651GM6-LTQReg1.03880.83991.13690.90691.24980.98080.50QReg23.54867.933223.546812.854326.487816.6496LTQReg (20%)0.84200.67351.23090.89517.90885.8414LTQReg (30%)0.65620.53650.80530.64481.24680.8983RWLTQReg0.40060.33220.79930.66581.14820.9557RMD-LTQReg0.47480.39190.52440.43020.56960.4662GM6-LTQReg0.76380.61760.88880.70171.22700.81420.75QReg17.251911.033624.135919.372528.269223.0904LTQReg (20%)1.06530.856412.705710.625919.988716.5240LTQReg (30%)0.88390.725810.95699.150217.418414.3180RWLTQReg2.74742.26298.47836.888513.616611.0773RMD-LTQReg0.66510.55250.74080.61220.82330.6760GM6-LTQReg1.35061.027913.563411.305521.486017.7558Table 2
RMSE and MAE values for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg at the three quantiles 0.25, 0.50, and 0.75, in the different contamination levels 20%, 30%, and 40% for the second experiment.
Sample sizeContamination rate10%20%30%TauMethodsRMSEMAERMSEMAERMSEMAE1000.25QReg7.600722.7113712.639555.4064915.295707.76377LTQReg (20%)0.964390.795511.129110.908032.575471.52490LTQReg (30%)0.837760.695870.940680.771941.105280.89317RWLTQReg0.612780.507900.703930.580180.761340.62801RMD-LTQReg0.614120.504720.635760.519190.608210.49545GM6-LTQReg1.004620.826151.046730.852081.076110.873030.50QReg7.034032.498619.505624.9003611.695556.27158LTQReg (20%)0.756410.615551.346681.081312.437781.87786LTQReg (30%)0.624990.511601.044470.854311.674201.36147RWLTQReg0.394080.320700.688700.562721.013280.82816RMD-LTQReg0.441340.357570.450010.363100.451310.36386GM6-LTQReg0.751620.612541.105780.902011.496761.223310.75QReg6.482702.9863810.870285.6874613.196356.99549LTQReg (20%)1.277771.046502.764382.252733.105302.52145LTQReg (30%)1.191030.984392.329181.911002.541142.09051RWLTQReg0.939360.778601.596551.313191.739521.43453RMD-LTQReg0.625200.514220.645710.529440.642470.52357GM6-LTQReg1.335481.088092.512962.059252.424491.996872000.25QReg8.096252.9380212.500055.3967115.543027.89287LTQReg (20%)0.976600.814001.121260.913532.589691.53480LTQReg (30%)0.855190.719150.936520.780481.108760.90673RWLTQReg0.658750.556840.696500.587540.769610.64687RMD-LTQReg0.621500.522970.618010.519200.622470.52132GM6-LTQReg1.037880.858811.052230.866221.080090.886720.50QReg7.467372.695869.446304.8971211.811016.27112LTQReg (20%)0.760730.627291.145120.925002.459821.97166LTQReg (30%)0.626270.521520.927520.765561.817341.48822RWLTQReg0.413740.344920.686050.571051.070190.89003RMD-LTQReg0.418870.347170.436970.360760.417050.34530GM6-LTQReg0.771880.636161.013860.829941.739161.427380.75QReg7.450193.9761410.920265.6266913.216337.00179LTQReg (20%)1.618941.346172.748052.276453.176202.61515LTQReg (30%)1.419701.195612.362341.984662.655882.22840RWLTQReg1.099510.933511.742171.480321.947451.65112RMD-LTQReg0.615260.518050.634960.533800.641020.53548GM6-LTQReg1.784271.466442.573932.148422.577852.16814Table1 and 2 show that the RMSE and MAE values for the QReg method at the different quantiles and different levels of contamination are the highest. That is, this method is more affected than the other methods by outlying data that fall in both x and y directions. On the contrary, we can see that the proposed method RMD-LTQReg has the lowest values of RMSE and MAE in most cases. This indicates that the performance of RMD-LTQReg is better than the others. On the other hand, the RWLTQReg has better performance than the other methods except the RMD-LTQReg. In addition, we can see when the contamination levels are 20% and 30%, the GM6-LTQReg performance is better than LTQReg (20%) in most cases and LTQReg (30%) in the 30% contamination level.In Figures6–9, we can see that the SE values for the parameters that estimated by the RMD-LTQReg method are the smallest in almost all cases, which indicates the performance of the RMD-LTQReg method is the best among all studied methods. Also, these figures show clearly that the QReg method has high SE leads to worse performance. In addition, the rest of the methods used showed close results in most cases and were varying in some other cases. Therefore, it could be argued that it is difficult to determine which one is better than others.Figure 6
SE values for the estimated parameters at each quantile and contamination level for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg, and M6 = GM6-LTQReg) in the first experiment whenn = 100.Figure 7
SE values for the estimate parameters at each quantile and contamination level for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg, and M6 = GM6-LTQReg) in the first experiment whenn = 200.Figure 8
The SE values for the estimated parameters at each quantile and contamination level for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg, and M6 = GM6-LTQReg) in the second experiment whenn = 100.Figure 9
The SE values for the estimated parameters at each quantile and contamination levels for all methods (M1 = QReg, M2 = LTQReg (20%), M3 = LTQReg (30%), M4 = RWLTQReg, M5 = RMD-LTQReg and M6 = GM6-LTQReg) in the second experiment whenn = 200.


## 5. Real Data Applications

In this section, the “Star Cluster CYG OB1” dataset is considered to verify the performance of our proposed methods.


### 5.1. Star Cluster CYG OB1

The Star Cluster CYG OB1 dataset was used by many researchers such as Rousseeuw and Leroy [24], Adrover et al. [9], and Neykov et al. [10]. This dataset contains 47 observations with one explanatory variable which is the logarithm of the effective temperature at the surface of stars. The independent variable is the logarithm of its light intensity. Rousseeuw and Leroy [24] presented that the scatterplot of this dataset shows two groups of observations. The first group includes the majority of data that contain 43 stars, whereas the second group includes the remaining four stars (the observations are 11, 20, 30, and 34). The observations 11, 20, 30 and 34 are classified as leverage points [24].In this example, we consider three quantiles (0.25, 0.50, and 0.75) to examine robustness of our proposed methods.Table3 presents the RMSE and MAE values for all proposed and existing estimation methods at each quantiles. It is clear to see that the QReg method has the highest RMSE and MAE values, whereas, the RMD-LTQReg method following by the RWLTQReg method has the better performance due to they have the smallest RMSE and MAE values, whereas the RMD-LTQReg method have detected the HLPs correctly. Also, we can see that the LTQReg (30%) is better than both of GM6-LTQReg and LTQReg (20%).Table 3
RMSE and MAE values for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg at the three quantiles 0.25, 0.50, and 0.75 for the Star Cluster CYG OB1 dataset.
Tau0.250.500.75MethodsRMSEMAERMSEMAERMSEMAEQReg0.93298330.61587960.57798430.466919730.70194060.5386939LTQReg (20%)0.36578600.29837840.35872010.292145750.48941140.3793147LTQReg (30%)0.33193830.26869210.29746920.240577490.37016670.2945234RWLTQReg0.22307260.17545450.16758430.139431820.24537460.2111627RMD-LTQReg0.19138850.15094000.12005520.096589390.18356360.1476853GM6-LTQReg0.42002580.33883330.38687420.316373120.35102180.2818068Figure10 shows the fitted residuals of regression quantiles for the existing and proposed methods. We can see that the QReg method is dramatically affected by the leverage points. Even though the RWLTQReg has lowest RMSE and MAE values in some cases, it is also affected by leverage points evident by trimming the observations that have high residuals, but it failed to trim leverage points. However, LTQReg (20%), LTQReg (30%), and GM6-LTQReg methods showed convergence in the chart and illustrated that these methods were also affected by the HLPs but better than QReg. The proposed method RMD-LTQReg shows a good performance due to its ability to trim the leverage points.Figure 10
Regression quantile for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg for the Star Cluster CYG OB1 dataset.


## 5.1. Star Cluster CYG OB1

The Star Cluster CYG OB1 dataset was used by many researchers such as Rousseeuw and Leroy [24], Adrover et al. [9], and Neykov et al. [10]. This dataset contains 47 observations with one explanatory variable which is the logarithm of the effective temperature at the surface of stars. The independent variable is the logarithm of its light intensity. Rousseeuw and Leroy [24] presented that the scatterplot of this dataset shows two groups of observations. The first group includes the majority of data that contain 43 stars, whereas the second group includes the remaining four stars (the observations are 11, 20, 30, and 34). The observations 11, 20, 30 and 34 are classified as leverage points [24].In this example, we consider three quantiles (0.25, 0.50, and 0.75) to examine robustness of our proposed methods.Table3 presents the RMSE and MAE values for all proposed and existing estimation methods at each quantiles. It is clear to see that the QReg method has the highest RMSE and MAE values, whereas, the RMD-LTQReg method following by the RWLTQReg method has the better performance due to they have the smallest RMSE and MAE values, whereas the RMD-LTQReg method have detected the HLPs correctly. Also, we can see that the LTQReg (30%) is better than both of GM6-LTQReg and LTQReg (20%).Table 3
RMSE and MAE values for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg at the three quantiles 0.25, 0.50, and 0.75 for the Star Cluster CYG OB1 dataset.
Tau0.250.500.75MethodsRMSEMAERMSEMAERMSEMAEQReg0.93298330.61587960.57798430.466919730.70194060.5386939LTQReg (20%)0.36578600.29837840.35872010.292145750.48941140.3793147LTQReg (30%)0.33193830.26869210.29746920.240577490.37016670.2945234RWLTQReg0.22307260.17545450.16758430.139431820.24537460.2111627RMD-LTQReg0.19138850.15094000.12005520.096589390.18356360.1476853GM6-LTQReg0.42002580.33883330.38687420.316373120.35102180.2818068Figure10 shows the fitted residuals of regression quantiles for the existing and proposed methods. We can see that the QReg method is dramatically affected by the leverage points. Even though the RWLTQReg has lowest RMSE and MAE values in some cases, it is also affected by leverage points evident by trimming the observations that have high residuals, but it failed to trim leverage points. However, LTQReg (20%), LTQReg (30%), and GM6-LTQReg methods showed convergence in the chart and illustrated that these methods were also affected by the HLPs but better than QReg. The proposed method RMD-LTQReg shows a good performance due to its ability to trim the leverage points.Figure 10
Regression quantile for QReg, LTQReg (20%), LTQReg (30%), RWLTQReg, RMD-LTQReg, and GM6-LTQReg for the Star Cluster CYG OB1 dataset.


## 6. Conclusions

In this paper, we proposed a new estimation method to overcome the impacts of leverage points in data. The new estimation method is called modified least trimmed quantile regression. In addition, we proposed three methods based on hard rejection weights that are used in reweighted least trimmed squares (Čížek [13]) to determine the trimming constant and to reduce the leverage point influence. In our proposed methods, the cutoff point of Gervini and Yohai [14] is employed for QReg. Moreover, Reweighted least trimmed, GM6 weights and robust Mahalanobis’ distance are developed for quantile regression.To investigate the performances of our proposed methods, a simulation study and real data are considered. The results indicate that the LTQReg has bad performance with data having leverage points due to it trims observations that have high residuals, whereas leverage points do not always have high residuals. Although, the RWLTQReg has good performance, evident by small RMSE, MAE and SE values, but it is not able to get rid of the leverage points. It is the same for the GM6-LTQReg that even though it is able to determine the trimming parameters, it is also affected by leverage points. From the results, it is clear to see that the RMD-LTQReg method is the best estimation method which can avoid the effect of leverage points.



---

*Source: 1243583-2020-06-12.xml*