metadata: authors: 0: author: Betti,Gianni
1: author: Molini,Vasco
2: author: Mori,Lorenzo
count: World
docna: 0: docna: New Algorithm to Estimate Inequality
            Measures in Cross-Survey Imputation : An Attempt to Correct
            the Underestimation of Extreme Values
docty: Policy Research Working Paper
abstracts: cdata!: This paper contributes to the debate on
            ways to improve the calculation of inequality measures in
            developing countries experiencing severe budget constraints.
            Linear regression-based survey-to-survey imputation
            techniques are most frequently discussed in the literature.
            These are effective at estimating predictions of poverty
            indicators but are much less accurate with inequality
            indicators. To demonstrate this limited accuracy, the first
            part of the paper discusses several simulations using
            Moroccan Household Budget Surveys and Labor Force Surveys.
            The paper proposes a method for overcoming these limitations
            based on an algorithm that minimizes the sum of the squared
            difference between a certain number of direct estimates of
            an index and its empirical version obtained from the
            predicted values. Indeed, when comparing the estimated
            results with those directly estimated from the original
            sample, the bias is negligible. Furthermore, the inequality
            indices for the years for which there are only model
            estimates, rather than direct information on expenditures,
            seem to be consistent with Moroccan economic trends.
display_title: New Algorithm to Estimate Inequality
            Measures in Cross-Survey Imputation : An Attempt to Correct
            the Underestimation of Extreme Values
listing_relative_url: /research/2022/04/33790853/new-algorithm-estimate-inequality-measures-cross-survey-imputation-attempt-correct-underestimation-extreme-values
topicv3: Poverty Measurement and Analysis m1326941 707,Inequality and Shared Prosperity m1327655 700,Statistical Surveys m1504545 3123
publishtoextweb_dt: 2022-04-20T00:00:00Z
dois: 10.1596/1813-9450-10013
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
author_keys: 0: author_key: 752858
1: author_key: 458815
model_extractions: Policy Research Working Paper 10013 New Algorithm to Estimate Inequality Measures in Cross-Survey Imputation An Attempt to Correct the Underestimation of Extreme Values _Gianni Betti Vasco Molini Lorenzo Mori_ Development Economics Development Data Group April 2022
Policy Research Working Paper 10013

# **Abstract**

This paper contributes to the debate on ways to improve the calculation of inequality measures in developing countries experiencing severe budget constraints. Linear regression-based survey-to-survey imputation techniques are most frequently discussed in the literature. These are effective at estimating predictions of poverty indicators but are much less accurate with inequality indicators. To demonstrate this limited accuracy, the first part of the paper discusses several simulations using Moroccan Household Budget Surveys and Labor Force Surveys. The paper proposes a method for overcoming these limitations based on an algorithm that minimizes the sum of the squared difference between a certain number of direct estimates of an index and its empirical version obtained from the predicted values. Indeed, when comparing the estimated results with those directly estimated from the original sample, the bias is negligible. Furthermore, the inequality indices for the years for which there are only model estimates, rather than direct information on expenditures, seem to be consistent with Moroccan economic trends.

This paper is a product of the Development Data Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at vmolini@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
A New Algorithm to Estimate Inequality Measures in Cross-Survey Imputation: An Attempt to Correct the Underestimation of Extreme Values Gianni Betti<sup>*</sup> Vasco Molini<sup>†</sup> Lorenzo Mori<sup>*‡</sup>

**_Keywords:_** Inequality indicators, Bias reduction, Survey-to-Survey Imputation, Moroccan HBS, Moroccan LFS **_JEL Classification:_** I32, C13 and E24

> *University of Siena. Department of economics and statistics. Email: gianni.betti@unisi.it

> †50x2030 Initiative. Email: vmolini@worldbank.org

> ‡University of Bologna. Department of Statistics P. Fortunati. Email: lorenzo.mori7@unibo.it
# **1 Introduction**

Collecting data to produce reliable and timely estimations of poverty requires elaborate and expensive surveys called Household Budget Surveys (HBSs). Only a few countries are able to collect data annually on income or expenditure to facilitate the estimation of poverty and inequality. Most National Statistical Offices (NSOs) worldwide collect these surveys only every four to five years. Therefore, producing reliable indices annually for monitoring poverty results remains quite challenging for many countries. To overcome this challenge, scholars have focused on developing methods to compare welfare indicators over time from surveys that are little comparable. These techniques, broadly known as Survey-to-Survey Imputation Techniques (SSITs), proved successful at predicting comparable poverty indicators, but as this paper argues, were less effective at predicting comparable inequality indicators.

SSITs are a derivative of the poverty map literature (Elbers et al., 2003; Tarozzi and Deaton, 2009, Christiaensen et al., 2012; Mathiassen, 2013). By imputing income onto censuses, poverty maps have been applied to many developing countries’ data to provide geographically disaggregated estimates of poverty. In addition to survey-to-census imputation, there has been a recent trend of survey-to-survey imputation, mapping from surveys with consumption data to those with other outcomes of interest, but lacking standard welfare aggregates (Elbers et al., 2004; Dabalen et al., 2014; Dang et al., 2019).

While this approach is well established for poverty estimates, a cursory overview of the literature shows that little attention is devoted to potential problems in obtaining accurate inequality measures with this method. Using actual data from a census of households in a set of rural Mexican communities, Demombynes and Hoogeveen (2007) show that correlations between the estimated and true welfare at the local level are highest for mean expenditure and poverty measures and lower for inequality measures. Douidich et al. (2016) obtain accurate estimates for quarterly poverty rates using a classic survey-to-survey imputation method that combines HBSs and Labor Force Surveys (LFSs). Using a log-linear regression to impute a completely missing variable (total expenditure) for certain years, they obtain reliable estimates for Head Count Ratio (HCR) with an exogenous poverty line. Krafft et al. (2019) impute consumption from HBSs onto LFSs and obtain information on poverty and inequality. Resulting measures of consumption, poverty, and inequality are similar across survey pairs, particularly for the data in Jordan and the Arab Republic of Egypt. These estimates are obtained based on the hypothesis that the model is time invariant, that is, the coefficients used to impute from one survey to another are stable over time (e.g., that _βt_ = _β_ ), and that the error term is homoscedastic and normally distributed. However, the concern is that incorrectly assuming normal errors or ignoring heteroscedasticity has the potential to introduce bias when estimating poverty or inequality. Specifically, in the prediction of inequality, the bias can be significant. Dang et al. (2019) provide an alternative method of simulating the final consumption imputation, which also seems extremely accurate when estimat2
ing poverty but does not address the estimation of inequality. To analyze poverty trends in Tunisia after 2010, Cuesta and Ibarra (2017) compare cross-survey micro imputation with macro projection techniques based on sectoral GDP, unemployment, and inflation. The SSIT used is an OLS regression, where the dependent variable of the model is the logarithm of annual household consumption per capita. Newhouse et al. (2014) provide evidence, on the other hand, illustrating that Survey-to-Survey Imputation can fail. They demonstrate that minor differences in the sampling scheme, sampling design, or structure of the answers/questions can produce inaccurate SSITs. Corral et al. (2021) show that there are more accurate methods (see also Rao and Molina, 2015) than poverty mapping to obtain estimates for small areas. Given that the SSITs are a derivative of this method, we argue their conclusions can also apply to SSITs.<sup>1</sup> Overall, the literature seems to indicate that the SSITs that are valid for predicting poverty can be equally valid for predicting inequality. By contrast, we argue that SSITs, by construction, tend to underestimate the predicted inequality, and to obtain accurate estimates, a different methodological approach should be chosen. The assumption of residuals’ normality distribution and the fact that standard SSITs are based on regression make them more accurate at predicting the central moments of a distribution (or transformations such as the poverty headcount) rather than the shape of the tails. This latter is crucial in predicting inequality. These models, however, tend to predict distribution compressed around the mean and, with thin tails, underestimate inequality. This point is not critical if the target parameters to be estimated are related with poverty, however, as shown by Schluter (2012), it is a crucial aspect of inequality measures. In this paper, we show that indicators of inequality obtained from fitted values can be biased if the variables common to the two surveys, as often happens, are not strongly correlated with the dependent one, that is, when the _R_<sup>2</sup> _ad j_<sup>of the regression is not high enough.</sup> To overcome these limitations, our approach is to estimate a semi-parametric model that combines the incorporation of the same covariates typically used by SSITs via the parametric component with a flexible fit to the data at hand via the non-parametric one. In general, these new techniques would seem particularly well suited to find a compromise between flexibility in terms of adaptation to the indicator to be predicted and consistency with standard survey-to-survey models. We tested this approach on a data-set recently used in Morocco (Douidich et al., 2016).

The paper is organized as follows: section 2 describes the data; section 3 frames the problem and includes simulations based on classic SSITs; section 4 presents the methodology used to produce results with the classical techniques; section; section 5 presents the proposed algorithm; section 6 describes the results of the new method; and section 7 concludes.

> 1In cases where the true population total or mean of an independent variable is known, we suggest using Small Area Estimation methods, in particular the M-quantile method developed by Chambers and Tzavidis (2006).

3
# **2 Survey Procedure**

Two sets of surveys were used, the HBS and the LFS. The HBS is carried out about every seven years, while the LFS is conducted every year. The HBS is used to determine total family expenditure, which we convert into expenditure per capita expressed in USD. The Moroccan HBS includes two different surveys: the 2000–2001 National Survey on Consumption and Expenditure (NSCE) and the 2006–2007 National Living Standards Survey (NLSS), which both measure household expenditure and are representative of the national and regional context and of urban and rural areas.

The NSCE covers 15,000 households and was administered between November 2000 and October 2000. The 2006–2007 NLSS, administered between December 2006 and November 2007, was smaller and covered only 7,200 households. The two questionnaires share many common modules, of which the most relevant address sociodemographic characteristics, habitat, expenditures, durable goods, education, health, and employment. The NSCE also has a module on transfers, subjective indicators of well-being, nutrition, and a module administered to the community to measure access to services.

Both surveys have the same structure. For urban areas, strata include the region, province, city size (large, medium, and small) and five types of housing. For rural areas, the strata are regions and provinces. The common points between the two surveys, however, end here. All the surveys conducted up to 2005, including the NSCE, have a master sample frame that is derived from the 1994 population census. The surveys conducted after 2005 are, instead, based on the 2004 census.

The 2001 NCSE follows a two-stage sampling process, while the 2007 NLSS follows a three-stage sampling process. In the first stage of NSCE, 1.250 Primary Sampling Units (PSUs) of approximately 300 households each were extracted from the 1994 population census. In the second stage, a dozen households per PSU were extracted randomly to constitute the final sample. In the first stage of NLSS, 1,848 PSUs of approximately 600 households each were selected from the 2004 population census. The second stage subdivided each PSU into twelve Secondary Sample Units (SSUs), representing about 50 households each, and then randomly selected six of the twelve SSUs from each PSU. In the third stage, a constant number of households was selected randomly from each SSU.

The LFS, which was first launched in 1976, follows a sampling process that is similar to that of the 2007 NLSS and shares a number of questions with the two censuses on which the population frame is based. The LFS questionnaires share only a few questions with the other two censuses, relating to sociodemographic characteristics, habitat, education, health and employment, but, significantly, do not share any information about expenditure.

4
# **3 A Critical Overview of Survey-to-Survey Regression Techniques**

In general, SSITs use a log-linear regression to estimate values for the parameter at year t, then use those values to predict values for the previous year _t − n_ if only the independent variables are available. This process is done under the hypothesis that the parameters are stable over the years. Often, this hypothesis is verified by estimating parameters at two non-consecutive years to illustrate that they are similar irrespective of the selected years. The regression usually uses individual or household expenditures as the dependent variable. With the right skew and in order to avoid normality, the variable is transformed into a logarithm and then transformed back to obtain the predicted values.

Regression is the best way to obtain predicted values only if all the assumptions are fulfilled and the selected dependent variables have a higher explanatory power versus the independent one. This last point is summarized in _R_<sup>2Often,articlesdonot</sup> _ad j_<sup>.</sup> report the _R_<sup>2althoughmultipleregressionsarealwaysused.Interactionorspecial</sup> _ad j_<sup>,</sup> transformation of the dependent variables is also common. Unfortunately, failure to report the _R_<sup>2</sup> _ad j_<sup>hinderstheabilitytochecktheaccuracyoftheregression.Giventhe</sup> connection between _R_<sup>2</sup> _ad j_<sup>and</sup><sup>_R_2wecanarguethatthefirstisatleastequaltoor—</sup> more likely — lower than the ones reported (which is often around 0.5 or less). A low _R_<sup>2</sup> can result from a selection that prefers a common set of variables from the two surveys — which is a key requirement for SSITs — over a high explanatory power.

Demographic variables have been identified in both surveys, such as age, sex, household size, or variables related to assets, such as the number of rooms per capita and measures of physical assets such as the asset index, among other variables. However, their choice might present two problems. First, it will be difficult to capture changes in household expenditure using only demographic variables (that change less rapidly). Second, there is often a weak correlation between dependent and independent variables.

As highlighted by Ketkar and Ketkar (1987), household sociodemographic characteristics are as important as determinants of expenditure patterns as price and income. The standard consumer behavior model states that the economic agent maximizes her utility subject only to relative prices and to income constraints. The two theories may both be valid, but if prices and income are excluded from the standard SSITs models, the sociodemographic variables alone may not adequately explain the individual or household consumption patterns. However, these conclusions must be verified for every data set and are impossible to determine a priori.

A low value for _R_<sup>2</sup> arising from insufficient correlated independent variables, combined with a logarithmic transformation, could also generate problems with establishing predictions. The logarithmic transformation compresses the tail of skewed and kurtotic variables, which effectively generate symmetric PDFs and, therefore, cause Gaussianlike errors. But the predicted values could be applied in the tails and, given the low 5
_R_<sup>2In addi-</sup> _ad j_<sup>, they could all be near the mean, with a sharp decrease in data variability.</sup> tion, logarithmic transformations are not immune to a re-transformation bias. There are various ways to solve this issue, but the most frequently used ones are based on scale correction of the predicted values. Both the relative Gini index and the relative Theil index are scale invariant,<sup>2</sup> which can render those corrections ineffective at reducing bias.

To summarize the discussion thus far, the regression to obtain a predictor is the right choice only if the _R_<sup>2</sup> _ad j_<sup>ishigh,alltheassumptionsarerespected,andthebiascaused</sup> by the log-transformation is negligible. In addition, it is important to bear in mind other potential sources of bias (Newhouse et al., 2014) arising from the differences between the survey design and the questionnaires.

# **4 Estimates of Biases Based on Moroccan Data**

In this section, using data from Morocco, we will show that the most common regression techniques can fail to accurately estimate three parameters: the Gini index, Theil index, and Head Count Ratio (HCR) with an endogenous poverty line equal to 60% of the median value of the regional expenditure per capita. We will compare estimates obtained with a direct estimator from the HBS at the rural/urban level in four different regressions. In all cases, the reported results will have a high negative skewed bias. The following regression techniques are used:

- Log-linear regression over two (split) data sets. We split the data set using a dummy (0= urban, 1= rural) variable, ran the regressions, and then computed the indices;

- Log-linear regression over the whole data set. The fitted values obtained are then split using the dummy (0= urban, 1= rural) variable and then the indices are estimated, both with ordinary least squares and feasible generalized least squares;

- Random forest regression over the split data set. With a log-transformation of the dependent variable;

- Quantile regression over the split data set with a log-transformation.

The fundamental problem with these regression techniques, even the robust ones, is that all fitted values will be near the mean value; therefore, variability will be reduced compared to the real regression. To attenuate this problem, we use two methods previously used in the literature (inter alia, Douidich et al., 2016 and Newhouse et al., 2014),

> 2The property of scale invariance states that inequality remains unchanged when all incomes increase by the same proportion. See Clementi et al. (2022)) for a discussion of differences between (relative) scale invariant and non-scale invariant (absolute) measures of inequality.

6
that produce reliable results when data from the census can be used or when the focus is on a poverty index with an exogenous poverty line. Table 1 reports the direct estimates of three inequality indices calculated from the 2000 and 2007 HBSs. The standard error reported in brackets is computed with a bootstrap estimator based on 500 iterations.

Table 1: Estimated Indicators by Sample Unit

|Id|20|00|20|07|
|---|---|---|---|---|
|nex|Rural|Urban|Rural|Urban|
|Gini|0.349|0.408|0.353|0.430|
||(0.006)|(0.005)|(0.006)|(0.006)|
|Theil|0.240|0.325|0.228|0.361|
||(0.017)|(0.016)|(0.012)|(0.015)|
|HCR|0.161|0.209|0.177|0.194|
||(0.005)|(0.004)|(0.007)|(0.006)|

_Source_ : Authors’ calculations based on Moroccan HBS.

Before moving to regression techniques, it is necessary to identify the variables that could be used as regressors. To do this, we must bear in mind that a bias could also arise from differences between questions and answers in the surveys.<sup>3</sup> To better understand the SSIT results, the trends of dependent and independent variables should be examined to predict trends for our indicators. The sample mean expenditure per capita in 2000 was 1 _,_ 752 _._ 96$ for rural areas and 3 _,_ 514 _._ 9$ for urban areas, which increased to 2 _,_ 265 _._ 9$ (+29%) and to 4 _,_ 053$ (+15%) respectively in 2007. The bulk of the dependent variables seem to have constant values over the years, with the notable exception of electricity. In rural areas, the proportion of families with electricity increased from 0.35 to 0.70.

The first model tested was the log-linear regression over the split data set. The results (see table [A.1]) are consistent with those presented by Douidich et al. (2016). Illiteracy and a low level of education have a negative impact on expenditure, while a higher number of rooms per capita has a positive impact, all other things being equal.

3Once the structure of questions and answers has been checked, we suggest also performing a graphical analysis of PDF and checking position and variability parameters, see Betti et al. (2018). We do not report this analysis for all the variables since the comparability has already been checked by Douidich et al. (2016) and we use the same set of variables. It is, however, important to underline that since 2015, Morocco has officially administered 12 regions. Previously, there were 16 regions. From that moment the new regions and the oldest ones are not directly comparable. In the interest of future comparability, we decided to use 14 regions, which was also decided by the Moroccan statistics department that unified the regions that had been merged. For more on this point, we recommend the official survey site. The only other variable that needs an explanation is the “low level of education,” with a value of 0 “up to the 1st cycle" and 1 for "over the 2nd cycle.

7
Furthermore, compared to those working in agriculture, those working in manufacturing or services are better off.

A comparison of the results from these models (table [A.2]) with those in table [1] clearly shows that the predicted inequality indicators are significantly biased downward. In a second stage, to increase the _R_<sup>2</sup> _ad j_<sup>, we regress the model over the whole data set,</sup> expecting an increase in the coefficient of determination. We obtain an _R_<sup>2</sup> _ad j_<sup>of 0.5649</sup> for 2000 and an _R_<sup>2</sup> _ad j_<sup>of0.4695for2007,indeedasignificantimprovementobserved</sup> only in urban areas. This moderate increase in the _R_<sup>2</sup> _ad j_<sup>for urban areas is caused by the</sup> introduction of one new regressor, the variable “milieu = urban (0) and rural (1).” However, the increase in _R_<sup>2</sup> _ad j_<sup>doesnotsignificantlyreducethebias.Following</sup> Newhouse et al. (2014), all parameters for the regression coefficients and the distributions of the error terms are estimated by feasible generalized least squares (FGLS<sup>4</sup> ). We do not consider the cluster-specific error terms since the clusters have changed between the two surveys and are not comparable.<sup>5</sup> Since the standard linear regression falls short of estimating our distributional indicators, we first use a random forest regression model followed by a quantile regression. We perform the first with _p/_ 3 = 5 knots and 500 trees. The _pseudoR_<sup>2</sup> is equal to 0.561 for 2000 and 0.489 for 2007. In the latter case, it is performed for each decile. Table [A.2] reports first the random forest results then those obtained for the quantile regression for q=0.5. Here, the goodness of fit is measured by _R_ 1( _q_ = 0 _._ 5) and is equal to 0.349 for 2000 and 0.289 for 2007. Similar results were obtained for the other deciles. These results highlight another problem encountered with the regression techniques. Usually, robust techniques perform very well due even in presence of outliers. Here, however, we are aiming to predict indices rather than a central tendency because the “extreme value” in the tails of the distributions are as important as those near the mean.

To sum up, the standard linear regression and the other two methods we tested drastically reduced the variability of the fitted values, which triggered a downward bias in the distributional indicators.<sup>6</sup> Hence, the proposed models failed to reproduce accurate inequality indicators because of the reduced data variability when the fitted values are used. In section 4, we propose an iterative algorithm that takes its cue from regression

> 4This method is also used because the studentized Breusch-Pagan test (BP = 307.78, df = 29, p-value _<_ 2 _._ 2 _e −_ 16) confirms the presence of heteroscedasticity in the OLS model.

> 5Nonetheless, we think that ignoring the cluster-specific error term is not the correct approach to obtain sound estimates for a single year. However, it is the only method we can use to avoid introducing another bias to the predicted values. There is persistent underestimation of the three parameters that is not improved by this method.

> 6The Ordinary Least Squares (OLS) method is based on the minimization of a function based on individual values. As is known, the confidence bands for the fitted lines reflect the minimum correspondence of the mean, which increases the error for the extreme values. This is acceptable if the goal is to predict a central tendency or something correlated with it but could be problematic if the indicators of interest are based on values in the tail of a distribution.

8
techniques but allows flexibility in the hypothesis of normality, changes the objective function, and does not use a transformation of the dependent variable.

# **5 The Proposed Algorithm**

The previous section indicates that SSITs based on regression techniques do not accurately predict inequality indicators. From this point on, we propose an algorithm that, using regression techniques as a point of departure, improves the results by removing hypotheses that can be too stringent and by changing the function to minimize. Our algorithm is based on the Constrained Optimization by Linear Approximation (COBYLA) method, a numerical optimization method developed by Powell (2007) and Conn et al. (1997) for constrained problems where the derivative of the objective function is not known.

Given a dependent variable, _yi_ , observed on _i_ = 1 _,..., n_ individuals, and a set of m covariates _Xn×m_ we predict _yi_ as a linear combination of _Xn×m_ , as follows: We then define the objective function to minimize as the sum of the square difference between the direct estimates of an index, _Ind_ , and its empirical version obtained for the fitted values, _Ine_ , that is: Where _J_ is the number of indices used. In this specific case, we used three indicators: the Gini index (G), Thiel index (T), and Head Count Ratio (HCR) with an endogenous poverty line. Equation [2] becomes: As the dependent variable is, by definition, non-negative, we introduce the constraint that the mean of the fitted values must be equal to the mean of the absolute value of the That is: where _y_<sup>_est_</sup> _i_ is obtained as follows: The other constraint used is that the absolute difference between the mean of the sampled value and the mean of the fitted ones must be less than 1, that is: 9
Hence, we add a lower bound and an upper bound to our parameters. Those bounds will be equal to ( _−Inf ,_ 0) for those variables that result in a negative value from the regression and to (0 _,_ + _Inf_ ) for all others. We perform a different regression for each area by checking where the signs agree between them. If there are differences–in some regressions some parameters could be either positive or negative–we opt for signs consistent with economic theory. The COBYLA<sup>7</sup> also requires an initial value from which to start. We use a constant, k, such that the value of k multiplied by the sum of mean values of covariates will be equal to the mean of the dependent variable. The regression parameters are not used since, if we start with this initial value, there is a difference in terms of scale in that some are denominated in tenths and others in thousandths, making the conversion particularly complex.

To estimate the coefficients’ variability, we use a bootstrap resampling method. First, we resample the data with replacement and the size of the resample equal to the size of the original data set. Then the parameters are computed from the resample from the first step. We repeat this routine _n_ = 100 times. Once the parameters and their variance are estimated and the results are verified, we rebuild estimates for the years in between the two surveys.

Estimation of temporally linked values assumes that _"everything is related to everything else, but near things are more related than distant things.”_<sup>8</sup> We estimate parameters for the initial year (2000) and the final year (2007) rather than for the years in between and use a weighted mean of _β_ 2000 and _β_ 2007. If _ωty_ represents the weights and _βty_ the parameters, where _ty_ indicates a generic year between _βmin_ and _βMAX_ and min and MAX are, respectively, the first and last year for which we have estimates, estimates are obtained by: Where: This type of weighting allows us to remove the hypothesis that regressors are stable and completely invariant over time. Finally, we obtain: From these values, we decided not to use an expansion estimator because even a small difference in the sampling scheme, and thus in the way the weights are obtained, could produce a bias. As shown by Newhouse et al. (2014), a change in the sampling scheme could be the reason that SSITs sometimes underperform. We estimate

> 7 _−_ 8 The stopping criteria used for this algorithm are a tolerance of 1 _e_ or a max evaluation of 100000 loop.

> 8First law of geography, Tobler (1970)

10
the variable distribution using a maximum likelihood estimator and then sample from the estimated distribution n-times a certain number of values to compute the parameter of interest. Therefore, since the Gini, Theil, and HCR indices are closely associated with the distribution of the variable, if we can estimate the exact distribution from the sample data, we can derive an accurate estimate of the indicators. To clarify, the classic log-normal distribution is used most often for consumption data. Assuming that the distribution and estimated parameters are correct, we will obtain an accurate estimate of the parameters. For example, we know that for a given log-normal distribution the related Gini coefficient is _G_ = 2Φ(<sup>_<u>σ</u>_</sup> 2<sup>)</sup><sup>_−_1.</sup> It is not possible to obtain a perfect estimate of the distribution as it is almost impossible to select the exact distribution function. Nonetheless, we argue that the bias will be minimal. The more closely the selected distributions fit the data, the greater the decrease in the margin of error. However, it is not strictly necessary for the sample scheme to be identical, only that both represent the same area for which we need estimates.

# **6 Results**

As mentioned in the section 4, we estimate the coefficients of the covariates through a linear regression and, in most cases, coefficient signs correspond to those previously estimated in table [A.1]. We prefer to use a linear regression and not a log one because we will not compute our algorithm through a transformation. The coefficient signs are key in determining the upper and the lower bounds of our algorithm. Thus, we will have positive values for all measures, except sex of household head, literacy, schooling level, and household size. These signs are consistent with findings in related literature.

Before discussing the results, it is important to further explore one aspect of the eq. [4]. As the Gini and HCR indices are between 0 and 1, and the Theil index is between 0 and log( _N_ ), the value of the direct estimate is known, from which the maximum of [4] can be easily computed. The reason for this is that for every index, the maximum will be equal to the maximum difference between its direct value and its maximum. The minimum will be chosen to maximize the single difference. This is a key step, since, once the maximum is computed, it is also easy to determine a relative error for each algorithm. The value of the estimated parameters, the number of iterations computed by each algorithm, the value of eq. [4] and its corresponding relative error are summarized in table [A.3].

The first three methods perform similarly, and the RE is generally the same. The last (urban areas for 2007) has the highest RE, about three times that for rural areas in the same year. It is difficult to say why this last has such a high RE. One plausible explanation is that mean expenditure per capita significantly increases from 2000 to 2007 in urban areas, but, because of the time-gap, covariates do not follow this pattern. Since the covariates vary little in urban areas between 2000 and 2007, the algorithm 11
cannot easily minimize the objective function within the constraints.

The estimated parameters presented in table [A.3] show the expected signs (imposed by the model) and coefficient sizes. It is also worth noting that their impact decreases in 2007; those increasing (in absolute terms) are mostly negative, suggesting that being illiterate or having a low level of education in urban areas becomes more disadvantageous in 2007 in economic terms. Using eq. [10] we compute the fitted value, from which we obtain the results reported in table 2. With _θ_<sup>ˆ</sup> as an estimated parameter and _θdir_ the corresponding direct estimates, it is possible to define the Absolute Relative Bias (ARB): This index facilitates comparison of all models in terms of bias (figure 1). This plot demonstrates that the only method that produces a low ARB is the one using the algorithm, with consistent bias reduction when compared with the other methods.

Table 2: Estimated Indices with Fitted Values, Algorithm Method

|Id|20|00|20|07|
|---|---|---|---|---|
|nex|Rural|Urban|Rural|Urban|
|Gini|0.354|0.425|0.356|0.442|
||(0.004)|(0.003)|(0.006)|(0.005)|
|Theil|0.233|0.323|0.230|0.343|
||(0.009)|(0.007)|(0.011)|(0.009)|
|HCR|0.175|0.207|0.167|0.268|
||(0.004)|(0.004)|(0.013)|(0.006)|

_Source_ : Authors’ calculations based on Moroccan HBS 12
<!-- Start of picture text --> Gini ARB% Theil ARB% HCR ARB%<br>8 Algorithm Algorithm & Algorithm<br>2R1 BRI - Jeri<br>® R2 % R2 @R2<br>8 R3 o |2R3 @ R3<br>"RE Oo) = R4 = R4<br>i=}vtg4°R5 =}o * RS | =JoO8 7° 85<br>_ g ie<br>e ef g<br>af<br>nN9° °7<br>=}nN<br>oO— oa<br>i=} c=} o<br>10 15 20 25 30 35 4.0 10 15 20 25 30 35 40 10 15 20 25 30 35 40<br>area Index index<br><!-- End of picture text -->
Table 4: Estimation of Rural and Urban Log-Normal Parameters

||Ru|20<br>ral|00<br>Urb|an|Ru|20<br>ral|07<br>Ur|ban|
|---|---|---|---|---|---|---|---|---|
||Dir.|Emp.|Dir.|Emp.|Dir.|Emp.|Dir.|Emp.|
||7.265|7.265|7.880|7.827|7.518|7.521|8.001|7.907|
|meanlog|(0.007)|(0.007)|(0.008)|(0.010)|(0.114)|(0.011)|(0.010)|(0.015)|
||0.595|0.604|0.696|0.753|0.615|0.608|0.716|0.928|
|sdlog|(0.006)|(0.006)|(0.007)|(0.006)|(0.009)|(0.008)|(0.009)|(0.013)|

_Source_ : Authors’ calculations based on Moroccan HBS.

Applying eq. [10], we obtain the figures shown in table [5], which reports the entire time series of inequality indicators between 2000 and 2007; 2000 and 2007 are calculated based on the sample data while the rest is imputed.

During the period considered, inequality slightly increases in urban areas while remaining relatively stable in rural ones. Growth in the early 2000s is less volatile than in the previous decade and posts substantially higher annual rates (Clementi et al., 2022). This growth, however, has not been accompanied by rapid economic modernization or significant labor market outcomes. Indeed, the Moroccan economy created an average of 115,000 jobs annually between 2000 and 2010. However, this could not absorb the annual increase in the working-age population (Lopez-Acevedo et al., 2021). Over the same period, the slight increase in inequality resulted from two counter-balancing trends: convergence of development across regions and increased intra-region inequality in some regions. Indeed, inequality increased mainly in the coastal, urbanized regions while decreasing in the — mainly rural — inner and eastern parts of the country (HCP, 2018).

As additional verification of the validity of this trend, we compared inequality measures in rural areas with average annual precipitation data.<sup>9</sup> As the Moroccan economy highly depends on agriculture, we expect a significant correlation between rainfall and inequality. In periods of drought, we expect inequality to increase since most of the agriculture is rainfed and only a few very developed areas (for example the Settat area) are equipped with modern irrigation systems that enable farmers to withstand shocks, which exacerbate rural inequality. In periods of abundant rainfall, we anticipate better performance overall of the agricultural economy and thus a decline in inequality. As expected, the correlation coefficients between the three indices and this data are, respectively, (G=-0.75, T=-0.67, HCR=-0.15). From these, we can deduce that the trend of the indices is consistent with rainfall patterns, confirming the validity of our hypothesis and the economic significance of our estimated results.

> 9https://climateknowledgeportal.worldbank.org/country/morocco/climate-data-historical

14
Table 5: Predicted Values (2000-2007)

|||Weig|hted we|ights|Log-norm|al dist.|
|---|---|---|---|---|---|---|
|||Gini|Theil|HCR|meanlog|sdlog|
||Urban|<sup>0.408</sup>|0.325|0.209|7.880|0.696|
|2000<sup>_∗_</sup>||<br>(0.005)|(0.016)|(0.004)|(0.010)|(0.008)|
||Rural|<sup>0.349</sup>|0.240|0.161|7.265|0.595|
|||<br>(0.006)|(0.017)|(0.005)|(0.007)|(0.006)|
||Urban|<sup>0.464</sup>|0.383|0.279|7.659|0.876|
|2001||<br>(0.003)|(0.009)|(0.003)|(0.057)|(0.004)|
||Rural|<sup>0.328</sup>|0.180|0.197|7.242|0.600|
|||<br>(0.002)|(0.003)|(0.004)|(0.021)|(0.007)|
||Urban|<sup>0.416</sup>|0.300|0.254|7.736|0.775|
|2002||<br>(0.003)|(0.005)|(0.003)|(0.002)|(0.003)|
||Rural|<sup>0.293</sup>|0.142|0.168|7.419|0.533|
|||<br>(0.002)|(0.002)|(0.003)|(0.011)|(0.010)|
||Urban|<sup>0.427</sup>|0.318|0.261|7.682|0.798|
|2003||<br>(0.003)|(0.006)|(0.003)|(0.004)|(0.003)|
||Rural|<sup>0.305</sup>|0.154|0.178|7.445|0.554|
|||<br>(0.002)|(0.002)|(0.003)|(0.007)|(0.009)|
||Urban|<sup>0.453</sup>|0.365|0.275|7.594|0.855|
|2004||<br>(0.003)|(0.008)|(0.003)|(0.014)|(0.002)|
||Rural|<sup>0.329</sup>|0.181|0.198|7.400|0.602|
|||<br>(0.002)|(0.003)|(0.003)|(0.005)|(0.003)|
||Urban|<sup>0.424</sup>|0.332|0.265|7.619|0.815|
|2005||<br>(0.001)|(0.002)|(0.001)|(0.002)|(0.112)|
||Rural|<sup>0.330</sup>|0.181|0.198|7.448|0.602|
|||<br>(0.002)|(0.003)|(0.003)|(0.004)|(0.006)|
||Urbn|<sup>0.415</sup>|0.300|0.254|7.544|0.777|
|2006|a|<br>(0.009)|(0.006)|(0.003)|(0.012)|(0.011)|
||Rrl|<sup>0.323</sup>|0.173|0.194|7.406|0.589|
||ua|<br>(0.002)|(0.002)|(0.004)|(0.007)|(0.003)|
||Urban|<sup>0.430</sup>|0.361|0.194|8.001|0.716|
|2007<sup>_∗_</sup>||<br>(0.006)|(0.015)|(0.006)|(0.001)|(0.009)|
||Rl|<sup>0.353</sup>|0.228|0.177|7.518|0.615|
||ura|<br>(0.006)|(0.012)|(0.007)|(0.114)|(0.009)|

_Source_ : Authors’ calculations based on Moroccan HBS and LFS. _Note_ : _year_<sup>_∗_</sup> index obtained from sample data 15
# **7 Conclusion**

This paper aims to contribute to the ongoing debate on ways to improve the accuracy and timeliness of welfare statistics in developing countries experiencing severe budget constraints. Only a few developing countries have the capacity to collect annual data on income or expenditure, therefore, indicators such as inequality or poverty rates can be computed in these countries only when Household Budget Surveys are available, about every four to five–and sometimes as many as seven–years. To overcome this problem, methods have been developed to compare these indicators over time from surveys that are little comparable. These techniques (SSITs) have proven effective at predicting poverty indicators, but are much less accurate when used for inequality indicators.

To illustrate this limitation, we conducted several simulations based on data from Moroccan Household Budget Surveys and Labor Force Surveys. Our results indicate that the predicted inequality measures are, at best, one-third smaller than those directly estimated from the data. Other regression methods were implemented, but none seems to significantly reduce the negative bias in the Gini, Theil, and HCR indices. Our theoretical explanation for this points to two main limitations of the Standard SSIT models based on linear regressions: the overly stringent assumption of the residuals’ normality distribution and the expectation that regression-based models predict distribution compressed around the mean and with thin tails. Unfortunately, the shape of the tails is crucial to correctly estimate inequality. Thus, almost by design, these models tend to produce estimates that are far below the correct values.

The method we propose is based on an algorithm that minimizes the sum of the squared differences between a certain number of direct estimates of an index and its empirical version obtained from the predicted values. With this algorithm, we reduce the bias and obtain results that are not systematically biased. When the estimated results are compared with those directly estimated on the original sample, the bias is negligible. Furthermore, the estimates of inequality indices for the years in which only labor force data are available seem to be consistent with Moroccan economic trends.

In the future, it would be interesting to explore the implementation and estimation of HCR for two or more areas where the poverty line stands at 60% of the national median. An alternative approach would be to identify the starting points to estimate the parameter and its variance. Other distributions of consumption should be tested in order to reduce the margin of error. The authors are working on the log-student-t distribution. For this paper, it is evident that bias reduction, realistic rebuilding of the values, and the economic interpretation of results represent a good starting point.

16