# Robust Regression-Ratio-Type Estimators of the Mean Utilizing Two Auxiliary Variables: A Simulation Study

Authors: Tolga Zaman; Emre Dünder; Ahmed Audu; David Anekeya Alilah; Usman Shahzad; Muhammad Hanif
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

Many authors defined the modified version of the mean estimator by using two auxiliary variables. These proposed estimators highly depend on the calculated regression coefficients. In the presence of outliers, these estimators do not give satisfactory results. In this study, we improve the suggested estimators using several robust regression techniques while obtaining the regression coefficients. We compared the efficiencies between the suggested estimators and the estimators presented in the literature. We used two numerical examples and a simulation study to support these theoretical results. Empirical results show that the modified ratio estimator performs well in the presence of outliers when adopting robust regression techniques.


---

## Body

## 1. Introduction

The classical ratio estimator is the most usual estimator ofY¯ when the correlation between study variable yi and auxiliary variables x1andx2 is positively high. However, when there are outliers in dataset, classical estimators perform poorly by decreasing their efficiency. As a result, the following studies in ratio estimators are available in the literature to lessen the detrimental impact of outlier data. In ratio estimators, Kadilar et al. [1] introduced using Huber-M estimate instead of least squares estimation (LSE). Noor-ul-Amin et al. [2] proposed to use Huber estimate, instead of LSE under double sampling. In ratio estimators, Zaman and Bulut [3] improved the estimators by applying robust regression coefficients. Zaman [4] provided estimators combining ratio estimators for Y¯ utilizing the ratio estimators given in Zaman and Bulut [3]. Subzar et al. [5] provided ratio estimators to predict Y¯ by using various robust regression techniques. Shahzad et al. [6] presented some ratio estimators to predict Y¯ by using of the Zaman and Bulut [3] ratio estimators for several cases of missing observations. Zaman and Bulut [7] presented ratio estimators by considering robust regression techniques and robust covariance estimations for stratified random sampling. Subzar et al. [8] presented various ratio estimators by considering robust regression tools. Ali et al. [9] provided robust regression-type estimators of the mean for simple random sampling. Grover and Kaur [10] improved the various ratio-type estimators by considering robust regression tools. In this paper, we are suggested to use of some robust regression estimates, instead of LSE, for the improved estimator of Y¯ by considering two auxiliary variables for simple random sampling.In the next section, we will go over ratio-type estimators forY¯ for simple random sampling, as well as their MSEs. In Section 4, we deduce the properties of the suggested estimators. Section 5 compares the efficiency of the estimators offered by Kadilar and Cingi [11] and the suggested estimators based on the MSEs. An empirical study utilizing two datasets and a simulation study are conducted, and we obtain satisfactory results, both theoretically and numerically. The numerical results are presented in Sections 6 and 7, respectively. We provide conclusion in the last section. In addition, the current study relies on robust estimates for two auxiliary variable studies for simple random sampling.


## 2. Traditional Ratio Estimator

Abu-Dayyeh et al. [12] provided the following estimator of Y¯ utilizing two auxiliary variables for simple random sampling and assuming that X¯1 and X¯2 are known:(1)y¯AD=y¯X¯1x¯1α1X¯2x¯2α2,where α1 and α2 are real numbers.Considering the ratio estimator presented in (1), Kadilar and Cingi [11] provided the following estimator:(2)y¯KC=y¯X¯1x¯1α1X¯2x¯2α2+b1X¯1−x¯1+b2X¯2−x¯2.The MSE equation of the estimator given in (2) was obtained as follows:(3)MSEy¯KC≅1−fnSy2+α1∗R1+B12Sx12+α2∗R2+B22Sx22−2α1∗R1+B1Syx1−2α2∗R2+B2Syx2+2α1∗R1+B1α2∗R2+B2Sx1x2,where R1=Y¯/X¯1, R2=Y¯/X¯2, α1∗=Syρx1x2ρyx1ρx1x2−ρyx2/R1Sx11−ρx1x22, and α2∗=Syρx1x2ρyx2ρx1x2−ρyx1/R2Sx21−ρx1x22.B1=Syx1/Sx12 and B2=Syx2/Sx22 are obtained by the LS estimate, f=n/N, Sx12 and Sx22 are the variances of population of x1i and x2i, respectively, and Syx1 and Syx2 are the covariance of population between yi and x1i and between yiand x2i, respectively [11].


## 3. Robust Regression Methods

Here, we describe some important and famous robust regression methods.


### 3.1. Huber-M Estimation

Huber [13] proposed an estimation by using different ρ functions known as M-estimators. This estimate is based on minimizing another function of outliers instead of error squares ei.The objective function of M-estimation is presented as(4)min∑i=1nρei,and is a symmetric function of outliers.Huber’s functionρ is designed as(5)ρe=e22,e≤m,me−m22,e>m.A derivative of the function presented in (5) is equal in the following equation:(6)φy=e,e≤m,msgne,e>m,where sgn. is sign function and specified as sgnx=−1e<m0e=m1e>m. For Huber’s estimate, m=1.345 and a constant value [14].


### 3.2. S Estimation

Rousseeuw and Yohai [15] proposed the regression estimates associated with M-scales to S estimation. S estimate is based upon the residual scale of M estimate. The estimate uses the residual standard deviation to tackle the weaknesses of the median. Function to minimize is given as follows:(7)min∑i=1nρyi−∑i=1nxijβσ^s,where 1/nL∑i=1nwiei2 in which L=0.199 and wi=ρui/ui2 [16].


### 3.3. Least Trimmed Squares Estimation (LTS)

In LTS estimate, initially squared error terms are sorted. Then, the sum of the firstφ of sorted error terms is taken, and function to minimize is given as follows:(8)min∑i=1φyi−β^0−β^1xi2,where φ=τ/2+1 and τ is the number of observation.


### 3.4. Least Median of Squares Estimation (LMS)

LMS estimate is improved by Rousseeuw and Leroy [17]. In LMS, the median of error squares is minimized. The following equation is minimized:(9)Min medianεi2.The estimate is robust against unusual observations in the direction of bothx and y, and its breakdown point is 0.5 [17]. For more detailed information about robust regression techniques, Zaman and Bulut [3] work can be investigated.


## 3.1. Huber-M Estimation

Huber [13] proposed an estimation by using different ρ functions known as M-estimators. This estimate is based on minimizing another function of outliers instead of error squares ei.The objective function of M-estimation is presented as(4)min∑i=1nρei,and is a symmetric function of outliers.Huber’s functionρ is designed as(5)ρe=e22,e≤m,me−m22,e>m.A derivative of the function presented in (5) is equal in the following equation:(6)φy=e,e≤m,msgne,e>m,where sgn. is sign function and specified as sgnx=−1e<m0e=m1e>m. For Huber’s estimate, m=1.345 and a constant value [14].


## 3.2. S Estimation

Rousseeuw and Yohai [15] proposed the regression estimates associated with M-scales to S estimation. S estimate is based upon the residual scale of M estimate. The estimate uses the residual standard deviation to tackle the weaknesses of the median. Function to minimize is given as follows:(7)min∑i=1nρyi−∑i=1nxijβσ^s,where 1/nL∑i=1nwiei2 in which L=0.199 and wi=ρui/ui2 [16].


## 3.3. Least Trimmed Squares Estimation (LTS)

In LTS estimate, initially squared error terms are sorted. Then, the sum of the firstφ of sorted error terms is taken, and function to minimize is given as follows:(8)min∑i=1φyi−β^0−β^1xi2,where φ=τ/2+1 and τ is the number of observation.


## 3.4. Least Median of Squares Estimation (LMS)

LMS estimate is improved by Rousseeuw and Leroy [17]. In LMS, the median of error squares is minimized. The following equation is minimized:(9)Min medianεi2.The estimate is robust against unusual observations in the direction of bothx and y, and its breakdown point is 0.5 [17]. For more detailed information about robust regression techniques, Zaman and Bulut [3] work can be investigated.


## 4. Suggested Estimators

In the presence of unusual observations, we propose utilizing the following 4 robust ratio estimators based on LTS, S, LMS, and Huber-M estimations instead of the ratio estimators stated in (2):(10)y¯pr1=y¯X¯1x¯1α1X¯2x¯2α2+b1LTSX¯1−x¯1+b2LTSX¯2−x¯2,(11)y¯pr2=y¯X¯1x¯1α1X¯2x¯2α2+b1SX¯1−x¯1+b2SX¯2−x¯2,(12)y¯pr3=y¯X¯1x¯1α1X¯2x¯2α2+b1LMSX¯1−x¯1+b2LMSX¯2−x¯2,(13)y¯pr4=y¯X¯1x¯1α1X¯2x¯2α2+b1HubMX¯1−x¯1+b2HubMX¯2−x¯2.The MSE equations of the suggested estimatorsy¯pri where i=1,2,3,4 are assumed to be the same as the expression for MSE in (3), but it is clear that B1 and B2 in (3) should be substituted by B1robk and B2robk, whose values as computed by LTS, S, LMS, and Huber-M estimates, respectively. The MSE equations for the suggested robust regression-ratio-type estimators belonging to LTS, S, LMS, and Huber-M estimates are obtained as follows:(14)MSEy¯pri≅1−fnSy2+α1∗R1+B1robk2Sx12+α2∗R2+B2robk2Sx22−2α1∗R1+B1robkSyx1−2α2∗R2+B2robkSyx2+2α1∗R1+B1robkα2∗R2+B2robkSx1x2,where B1robk and B2robk are obtained from LTS, S, LMS, and Huber-M estimates, respectively (k = LTS, S, LMS, and Huber-M).


## 5. Efficiency Comparisons

We compare the MSE equation for the traditional estimator, presented in (3), with the MSE equations for the suggested robust regression-ratio-type estimators, given in (14), to derive the conditions for which the suggested estimators will perform better than traditional estimator for two auxiliary variables for simple random sampling.(15)MSEy¯pri<MSEy¯KC;i=1,2,3,4,α1∗R1+B1robk2Sx12+α2∗R2+B2robk2Sx22−2α1∗R1+B1robkSyx1−2α2∗R2+B2robkSyx2+2α1∗R1+B1robkα2∗R2+B2robkSx1x2<α1∗R1+B12Sx12+α2∗R2+B22Sx22−2α1∗R1+B1Syx1−2α2∗R2+B2Syx2+2α1∗R1+B1α2∗R2+B2Sx1x2,B1robk−B1Sx12B1robk+B1+2Sx12α1∗R1−2Syx1+2Sx1x2α2∗R2+B2robk−B2Sx22B2robk+B2+2Sx22α2∗R2−2Syx2+2Sx1x2α1∗R1+2Sx1x2B1robkB2robk−B1B2<0.The suggested robust regression-ratio-type estimators in (10)–(13) perform better than the ratio estimator in (2) when condition (15) is satisfied.


## 6. Numerical Illustrations

In this part, we performed numerical examples on two real datasets. The first dataset (Education) was collected to model the expenditure for public education [18]. The second dataset (Crime) was used to predict the violent crimes in states [19]. These datasets were previously used for investigating the robust techniques in the literature. Education dataset exists in robustbase package in R software [20].We employed four robust estimators: LTS, LMS, S, and Huber-M, respectively. We compared the efficiency of robust techniques with LSE. While assessing the performance of the estimators, we used the efficiencies relative to LSE. We used R programming language in the implementation phase [21]. We utilized MASS and robustreg packages for robust regression analysis [20, 22]. We obtained the MSE values for each estimator for performance evaluation.The definitions of the variables existing in datasets are shown in Table1. The datasets contain two auxiliary and one study variables. Table 2 indicates the descriptive statistics for the real datasets.Table 1
Description of the auxiliary and study variables for datasets.
DatasetVariableDescriptionEducationx1Number of residents per thousand residingx2Per capita personal incomeyPer capita expenditure on public educationCrimex1∗Number of murders per 1,000,000 peoplex2∗Percentage of the people who are single parentsy∗Violent crimes per 100,000 peopleTable 2
Descriptive statistics of the variables for datasets.
DatasetVariableMeanSTDMinMaxa1∗a2∗Educationx1657.800145.016322909−0.4050.089x24675.120644.50634485889y284.60061.340208546Crimex1∗8.72710.7181.60078.500−0.149−2.085x2∗11.3252.1218.40022.100y∗612.843441.10082.0002922.000In Table3–6 , the covariance and correlation matrices of the variables are given for both datasets. The high positive correlation values satisfy the condition of the applicability for the ratio estimators.Table 3
Covariance matrix for Education dataset.
DatasetVariablex1x2yEducationx121029.75558128.6802865.327x258128.678415388.31024048.702y2865.32724048.7003762.612Table 4
Covariance matrix for Crime dataset.
DatasetVariablex1∗x2∗y∗Crimex1∗114.86619.5294189.516x2∗19.5294.501785.012y∗4189.516785.012194569.495Table 5
Correlation matrix for Education dataset.
DatasetVariablex1x2yEducationx110.6220.322x20.62210.608y0.3220.6081Table 6
Correlation matrix for Crime dataset.
Variablex1∗x2∗y∗Crimex1∗10.8590.886x2∗0.85910.839y∗0.8860.8391We calculated the Mahalanobis distances for checking the existence of potential outliers. For each observation “i,” Mahalanobis distances are calculated as follows:(16)MAHDİSTi=xi−LC−1xi−LT,where “L” shows the location matrix and “C” indicates the covariance matrix. The cut-off value is χp,1−α/22 where p is the number of variables. The α term represents the error level. The observation is considered as the potential outlier when exceeds the cut-off value.We used minimum covariance determinant (MCD) estimators for avoiding the masking effect [23]. Figures 1 and 2 represent the Mahalanobis distance plots for the real datasets. The cut-off value is given with a straight line in plots. Clearly, we can see that Education and Crime datasets include some potential outliers. Also, these two datasets were used to evaluate the robustness issues in the literature.Figure 1
Mahalanobis distance plot for Education dataset.Figure 2
Mahalanobis distance plot for Crime dataset.Table7 indicates the regression coefficients which were obtained using the robust estimates and LSE. The coefficients are rather different from LSE for robust estimates. Although there is a positive correlation between x1 and y in Education dataset, B1 is negative. This case shows the corruptive effect of outliers in dataset. Robust estimates adroitly overcome this problem, and all regression coefficients are positive for each robust estimate.Table 7
Regression coefficients for each estimator.
DatasetEstimatorsB1B2EducationLSE−0.0390.063LTS0.1160.029S0.1000.034LMS0.1180.029Huber-M0.0420.045CrimeLSE25.99961.607LTS43.65571.956S46.21057.910LMS50.29358.780Huber-M22.23583.478We use the MSE values of the conventional and suggested robust regression-ratio-type estimators, as specified in Sections2 and 4, to calculate the relative efficiency of each suggested robust regression-ratio-type estimator in (10)–(13) in comparison to the classical estimators in (2), utilizing the formulae:(17)RE=MSEy¯priMSEy¯KC,i=1,2,3,4.Table8 denotes the performance results of the estimators. According to efficiencies, the suggested robust regression-ratio-type estimator is better than the estimator presented by Kadilar and Cingi [11]. Especially the suggested robust regression-ratio-type estimators based on S and LTS estimates produced the lowest MSE values for Education and Crime dataset, respectively. The classical estimator has the highest MSE value when comparing with four robust regression-ratio-type estimators in both two datasets. These results are not surprising because condition (15) is satisfied.Table 8
Theoretical results for the efficiency values of each suggested robust regression-ratio-type estimator with respect to the traditional estimator.
DatasetEstimatorREEducationy¯KC1y¯pr10.601y¯pr20.525y¯pr30.618y¯pr40.749Crimey¯KC1y¯pr10.621y¯pr20.767y¯pr30.764y¯pr40.879


## 7. Simulation

A simulation study is performed to evaluate the efficiencies of the suggested robust regression-ratio-type estimators. Epilepsy dataset is used for the simulation [24]. The dataset include two auxiliary and one study variable. There are n=59 observations in epilepsy dataset. The purpose of this dataset is to predict the epilepsy attacks. Epilepsy dataset exists in “robustbase” package of R programming language.The description of the variables for epilepsy dataset is shown in Table9. Sum of Y is the study, and the other variables are auxiliary variables.Table 9
Description of variables.
VariableDescriptionAgeAge values of the patientsBaseNumber of epileptic attacks recorded which is prior to randomizationSum of YTotal number of epilepsy attacksFigure3 demonstrates the Mahalanobis distance plot for epilepsy dataset. The distances are obtained similar to the previous application section. Obviously, we can see that this dataset contains some possible outlier observations.Figure 3
Mahalanobis distance plot for epilepsy dataset.We randomly selected sample from the datasets for 10000 times randomly and estimated the population means using the traditional and proposed estimators. We computed the MSE equation as follows:(18)MSEY¯^=110000∑i=110000Y¯^i−Y¯2,where Y¯^i indicates the estimation of mean for i=1,2,…,9999,10000 and Y¯ represents the priorly known population mean of the study variable. In Table 10, we reported the MSE ratios of the suggested robust regression-ratio-type estimators with respect to the traditional estimator for each dataset. These values are obtained using (18). Numerical results were conducted for the sizes of sample n=10,20,30,40. Computations were run in R software.Table 10
Simulation results for the efficiency values of each suggested robust regression-ratio-type estimator with respect to the traditional estimator.
Sample sizeEstimatorsREn = 10y¯KC1y¯pr10.44y¯pr20.467y¯pr30.438y¯pr40.682n = 20y¯KC1y¯pr10.503y¯pr20.53y¯pr30.512y¯pr40.719n = 30y¯KC1y¯pr10.583y¯pr20.577y¯pr30.59y¯pr40.763n = 40y¯KC1y¯pr10.682y¯pr20.699y¯pr30.691y¯pr40.816Table10 denotes the simulation results. According to results, the suggested robust regression-type estimators apparently outperform the estimator presented in Kadilar and Cingi [11] in all sample sizes. In most cases, the suggested robust regression-type estimator based on LTS estimate has the lowest MSE value. As the sample size grows, all estimators produce lower MSE values. Generally, the suggested regression-type estimators based on robust techniques perform better than traditional estimator and overcome the outliers problem. These simulation findings support the theoretical results in Table 8.


## 8. Discussion

Tables8 and 10 clearly show that proposed robust regression-ratio-type estimators for estimating Y¯ by utilizing outliers data for simple random sampling that are more efficient. The estimators of (10)–(13) provide lower MSEs than the MSE of traditional ratio estimator of (3). This means that the estimators of (10)–(13) show high performance than the estimator presented by Kadılar and Cingi [11]. These results have been demonstrated theoretically and supported by both empirical and simulation results.


## 9. Conclusion

Traditional ratio estimators suffer from the outliers because of the distorting effect. In this study, we proposed robust regression-ratio-type estimators using several robust estimates to handle the robustness task for the estimator proposed by Kadılar and Cingi [11]. We aimed to improve the performance of the suggest estimators by adopting robust coefficients. We used two real dataset examples containing possible outliers for comparing the suggested robust regression-ratio-type estimators with traditional estimator. According to our findings, the suggested robust regression-ratio-type estimators based on all robust techniques have lower MSE values when comparing with the traditional estimator. Numerical results demonstrate that the suggested robust regression-ratio-type estimators provide more efficient results than the traditional estimator. We hope that in the future we will expand the estimators presented here to other sampling designs.



---

*Source: 6383927-2021-09-06.xml*