metadata: authors: 0: author: Beltram,Theresa
1: author: Dang,Hai-Anh H.
2: author: Sarr,Ibrahima-000535387
3: author: Verme,Paolo
count: World
docna: 0: docna: Estimating Poverty among Refugee Populations
            : A Cross-Survey Imputation Exercise for Chad
docty: Policy Research Working Paper
subtopic: Inequality,Educational Sciences,Demographics,Food Security,Nutrition,Gender and Development
teratopic: Poverty Reduction,Education,Gender,Agriculture,Health, Nutrition and Population
abstracts: cdata!: Household consumption surveys do not
            typically cover refugee populations, and poverty estimates
            for refugees are rare. This paper tests the performance of
            cross-survey imputation methods to estimate poverty for a
            sample of refugees in Chad, by combining United Nations High
            Commissioner for Refugees survey and administrative data.
            The proposed method offers poverty estimates based on
            administrative data that fall within a 95 percent margin of
            poverty estimates based on survey consumption data. This
            result is robust to different poverty lines, sets of
            regressors, and modeling assumptions of the error term. The
            method outperforms common targeting methods, such as proxy
            means tests and the targeting method currently used by
            humanitarian organizations in Chad.
repnme: repnme: Estimating Poverty among Refugee
            Populations : A Cross-Survey Imputation Exercise for Chad
display_title: Estimating Poverty among Refugee
            Populations : A Cross-Survey Imputation Exercise for Chad
listing_relative_url: /research/2020/04/31987874/estimating-poverty-among-refugee-populations-cross-survey-imputation-exercise-chad
topicv3: Food Security m1326439 199,Poverty m1326687 695,Demographic Change m1327040 1448,Education m1327048 470,Gender m1327621 615
publishtoextweb_dt: 2020-04-27T00:00:00Z
historic_topic: Poverty Reduction,Education,Gender,Agriculture,Health, Nutrition and Population
origu: Strategy, Analytics, Fin. Solut. (GTFSA)
no_of_pages: 32
author_keys: 0: author_key: NA
1: author_key: 747967
2: author_key: 1662668
3: author_key: 439872
model_extractions: Policy Research Working Paper 9222 Estimating Poverty among Refugee Populations A Cross-Survey Imputation Exercise for Chad _Theresa Beltramo Hai-Anh H. Dang Ibrahima Sarr Paolo Verme_ Fragility, Conflict and Violence Global Theme & Development Data Group April 2020
Policy Research Working Paper 9222

# **Abstract**

Household consumption surveys do not typically cover refugee populations, and poverty estimates for refugees are rare. This paper tests the performance of cross-survey imputation methods to estimate poverty for a sample of refugees in Chad, by combining United Nations High Commissioner for Refugees survey and administrative data. The proposed method offers poverty estimates based on administrative data that fall within a 95 percent margin of poverty estimates based on survey consumption data. This result is robust to different poverty lines, sets of regressors, and modeling assumptions of the error term. The method outperforms common targeting methods, such as proxy means tests and the targeting method currently used by humanitarian organizations in Chad.

This paper is a product of the Fragility, Conflict and Violence Global Theme and the Development Data Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/ prwp. The authors may be contacted at pverme@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Estimating Poverty among Refugee Populations: A Cross-Survey Imputation Exercise for Chad**

_Theresa Beltramo Hai-Anh H. Dang Ibrahima Sarr Paolo Verme_<sup>1</sup>

**JEL classifications** : C15, F22, I32, O15, O20.

**<mark>Keywords</mark>** <mark>: Refugees; Forced displacement; Targeting; Poverty; Chad.</mark>

1 * Theresa Beltramo: United Nations High Commissioner for Refugees, Rue de Montbrilliant 92, Geneva 1201, Switzerland (email: beltramo@unhcr.org); Hai-Anh Dang: DECAT- Analytics and Tools Team-The World Bank, 1818 H St. N.W. Washington, D.C. 20433 (e-mail: <u>hdang@worldbank.org); Ibrahima Sarr: United Nations High Commissioner for Refugees (e-mail: sarri@unhcr.org); Paolo Verme:</u> <mark>Fragility, Conflict, Violence - T</mark> he World Bank, 1818 H St. N.W. Washington, D.C. 20433 (email: pverme@worldbank.org <mark>). The authors are grateful to Andrew Dabalen, Giacomo De Giorgi, Kristen Himelein, Aly Sanoh, and participants to the American Economic Association’s session on “Comparing Poverty of Refugees and Their Hosts” (San Diego, January 5</mark><sup>th</sup> <mark>, 2020) for helpful comments on previous versions. This work is part of the program ``Building the Evidence on Protracted Forced Displacement: A Multi-Stakeholder Partnership''. The program is funded by UK aid from the United Kingdom's Department for International Development (DFID), it is managed by the World Bank Group (WBG) and was established in partnership with the United Nations High Commissioner for Refugees (UNHCR). The scope of the program is to expand the global knowledge on forced displacement by funding quality research and disseminating results for the use of practitioners and policy makers. We further thank DFID for additional funding through its Knowledge for Change (KCP) program. This work does not necessarily reflect the views of DFID, the WBG or UNHCR.</mark>
# **I. Introduction**

UN General Assembly Sustainable Development Goal 1— _End poverty in all its forms_ by 2030— explicitly pledges that “no one will be left behind.” To achieve this goal, the availability of highquality household consumption surveys is essential, and it is equally important for these surveys to be inclusive and cover marginal populations, such as refugees and internally displaced persons (IDPs). Unfortunately, household consumption surveys rarely include forcibly displaced populations, despite that these populations are among the most vulnerable and deprived. They lack fundamental rights such as freedom of movement and the right to work, have eroded human and physical capital, and face more frequent shocks than surrounding host communities do.

This is a significant and growing challenge, particularly in Sub-Saharan Africa. The global number of forcibly displaced persons grew from 43.3 million in 2009 to 70.8 million in 2018. Among them, there are 25.9 million refugees, 3.5 million asylum seekers, and 40.3 million IDPs. Almost four out of five refugees live in a country neighboring their home country, and some 84% of them live in developing countries. Sub-Saharan Africa hosts around one-third of the world’s refugee population and witnessed an increase of 1.1 million refugees in 2017, which represents a 22% increase from 2016. In 2018, Sub-Saharan African countries represented half of the 10 countries with the highest refugee population relative to the national population and six of the 10 countries with the highest numbers of IDPs. The region also has the highest poverty rates in the world, and data for the region are typically scarce or of low quality. Therefore, measuring poverty among displaced populations in Sub-Saharan Africa is a particularly important task that is severely hampered by missing household consumption data.<sup>2</sup> <mark>There is an established literature in statistics that has developed methods to impute missing data (see Little and Rubin (2002) for a review of these methods). Survey-to-survey imputation methods have been widely employed in economics to estimate household welfare trends across time periods in the context of repeated cross-section or panel data (e.g., Dang and Lanjouw 2018), across geographical areas in the context of poverty mapping exercises (e.g., Elbers et al. 2003), or across</mark>

> 2 <mark>Missing data issues are not a problem limited to displaced populations, but can emerge because of lack of survey data on a particular topic of interest, population group, or time period. These issues can also be caused by sampling errors, incomplete data due to unit or item nonresponse, data input errors, or post-survey data manipulations such as top-coding or censoring.</mark>

2
<mark>types of surveys such as consumption and labor force surveys (e.g., Douidich et al. 2016). Newhouse</mark> et al. (2014 <mark>) and Dang et al. (2019) offer recent summaries of previous imputation studies that highlight the main advantages, debate different approaches, and provide useful insights about welfare imputation practices. More recently, Dang and Verme (2019) is the first study to propose the use of a cross-survey imputation method in the context of the Syrian refugees living in Jordan, using survey and administrative data provided by the</mark> United Nations High Commissioner for Refugees <mark>(UNHCR). Their findings suggest that in the absence of actual household consumption survey data, their cross-survey imputation method can provide a secondbest alternative to address the data challenge.</mark> <mark>In this paper, we make several new contributions to the nascent literature on poverty measurement for refugees. First, we apply the cross-survey imputation method to a sample of refugees based in Chad. Since this country is in Sub-Saharan Africa and one of the poorest countries in the world, its differences from Jordan in both geographical location and income levels represent a new setting to apply and further validate this imputation method. Second, we exploit a richer and more diverse set of data than the Jordan study, which includes r</mark> egistration data, census-type targeting data, and a household consumption survey. We also rigorously examine the imputation method against different poverty lines, including the food poverty line, the national poverty line, the international poverty line, and various other simulated values. Finally <mark>, in addition to the poverty imputation exercise, we test the performance of the proposed method for targeting purposes. In particular, we compare the targeting performance of our method with the targeting method currently used in Chad to administer cash assistance to refugees as well as with the global experience.</mark> <mark>The estimation results indicate that the limited set of variables available in the UNHCR registration data predict household consumption (welfare) reasonably well. Estimates from the three sets of data available for the analysis produce similar welfare figures. Th</mark> e <mark>current targeting strategy in Chad, which is used jointly by the N</mark> ational Commission on the Welcoming and Resettlement of Refugees <mark>(CNARR), UNHCR, and World Food Programme (WFP), is accurate in predicting household welfare. However, our results suggest that this targeting strategy could be further improved by reducing the inclusion and exclusion errors. If these encouraging results are replicated</mark> 3
<mark>in other contexts, poverty predictions for refugees can be expanded at scale, with good prospects for the improvement of targeted programs.</mark> <mark>The poverty estimates used in this paper do not reflect the official poverty estimates monitored by the government and the international community. The interest of this paper is to test a cross-survey imputation methodology, and, for this purpose, we use a subsample of UNHCR refugee data that are not nationally representative of the refugee population in Chad. By contrast, the official poverty statistics require national consumption surveys conducted by the national statistical office with samples that are nationally representative. At the time of writing this paper, these national data were not available, but they are expected to be published before the end of 2020. This will provide another opportunity to validate the cross-survey imputation method proposed in this paper.</mark> <mark>The paper is organized as follows. Section II outlines the country context. Section III presents the data and analytical framework. T</mark> he estimation results are presented in section IV, and section V evaluates the <mark>targeting strategy</mark> used in Chad and our targeting method in light of the global experience. Section VI discusses the limitations of the study, and section VII concludes.

# **II. Country Context**

Chad is one of the poorest countries in the world. According to the latest national consumption survey, which was administered in 2011, 29% of the population falls below the food poverty line and 47% falls below the national poverty line (World Bank 2018). The past decade has been a decade of instability for Chad, with negative consequences on household well-being. Pe <mark>r capita gross domestic product decreased by 15% between 2015 and 2017, from an average of US$963 in 2015 to US$823 in 2017 (in 2010 purchasing power parity (PPP)). In terms of overall development, Chad is</mark> ranked 187 of 189 countries on the Human Development Index (World Bank 2019). Due to these challenges, Chad struggled to meet many of the Millennium Development Goals in 2015, and barring unforeseen economic growth or great increases in official development assistance, the country is not likely to meet many of the Sustainable Development Goals objectives set for 2030.

4
Despite the current negative economic downturn, Chad continues to host a high number of refugees and is among the countries that top the world’s list in this respect (Table A1). Chad is the 10th largest host country for refugees in the world and the fifth largest host in the Africa region after Ethiopia, Kenya, Uganda, and the Demographic Republic of Congo. Chad’s refugee population represents a significant portion of the national population—about 3%. The number of forcibly displaced persons increased from 474,478 in 2015 to 667,586 as of March 2019, of which about 69% were refugees or asylum seekers.<sup>3</sup> Of the 459,809 current refugees and asylum seekers, the majority are Sudanese refugees (74%) living in the eastern part of Chad; 21% are Central African Republic refugees living in southern Chad; a much smaller number of Nigerian refugees (about 2%) are living in the Lake Chad Basin. The situation is further complicated by the large population of IDPs in the Lake Chad region, estimated at 165,313 at end of 2018 (UNHCR 2018). Map 1 shows the locations of the refugee camps in Chad.

# **III. Methodology and Data**

# **<mark>Methodology</mark>**

<mark>The methodology used in this paper relies on the cross-survey imputation framework, which was first introduced by Elbers</mark> et al. <mark>(2003).</mark><sup>4</sup> <mark>Most recently, Dang</mark> et al. <mark>(2017) build on this literature to propose a model that imposes fewer restrictive assumptions and offers an explicit formula for estimating the poverty rate and its variance. Three advantages of the modifications introduced by Dang</mark> et al. <mark>(2017) are (i) the variance formula, which is simple and in line with the recent statistical literature; (ii) it can accommodate complex design sampling; and (iii) the framework remains applicable to two surveys with different designs. Finally, the approach allows for different modeling methods, including the standard linear regression model, its variant with a flexible specification of the empirical distribution of error terms, a logit model, and/or a probit model.</mark>

> 3 UNHCR uses the term “people of concern” to describe those who are forcibly uprooted from their homes, including asylumseekers, refugees, stateless persons, the internally displaced, and returnees.

> 4 See also <mark>Tarozzi (2007) and Mathiassen (</mark> 2009) <mark>for further improvements and adaptation of this approach (e.g., by estimating the standard errors in a different way).</mark>

5
Let xj be a vector of characteristics that are commonly observed between the two surveys, where _j_ indicates survey type, with 1 the base survey and 2 the target survey. Let us assume that the welfare indicator is a function of household and individual characteristics ( ): jj xx where is the welfare indicator (consumption per capita per month), is a vector of parameters, is cluster random effects, and jj is the idiosyncratic error term. jj yy ββ ccjj jj μμ εε The framework proposed by Dang et al. (2017) is based on two assumptions. The first assumption (Assumption 1), which is critical for poverty imputation, states that measurement of household characteristics in each sample of data is a consistent measure of the characteristics of the whole population. In other words, it stipulates that the two surveys are representative of the same target population. In our context, the two surveys represent the same population of refugees, and they were conducted approximatively at the same time. Therefore, we do not expect major issues with this first assumption. However, we will conduct means difference tests on the observed overlapping variables between the target data and base data to ensure that this is the case. The second assumption states that changes in between the data collection periods of the two data sets can capture the change in welfare over the period (Assumption 2). Since data collection for jj xx the two data sets we use refers to the same year, there is no need to test Assumption 2. Under these two assumptions, the imputed welfare is Dang et al. (2017) propose different imputation methods for parameter estimation. The first method relies on the assumption of the normal distribution for the two error terms (μcj and εj are uncorrelated and μcj ∕xj μcj) and εj ∕xj εj)). Hereafter, this method is referred to as the normal linear regression model. An alternative method proposed is the empirical error ~NN(0, σ ~NN(0, σ method, which assumes no functional form for these error terms and uses instead the empirical distribution to estimate the parameters. Dang et al. (2017) also propose two other alternative 6
methods—the Probit Model and the Logit Model— which are more restrictive and model poverty status (poor and nonpoor) instead of consumption expenditure.

Once the parameters are estimated, the welfare indicator, which is household consumption per capita, is obtained as follows: The imputed poverty rate and its variance are then estimated as: These poverty estimates are unbiased estimates of the parameters of interest (Dang et al., 2017) and outperform in terms of prediction accuracy those proposed by the proxy means testing literature, which typically omits the error terms υ1 + ε1, leading to biased estimates of the welfare indicator.

# **Data**

In its mandate to protect displaced persons in host countries, the UNHCR collects data to track refugees and other populations of interest, better monitor these populations, and deliver assistance and services. In the framework of this study, we use three sets of data collected by the UNHCR (Table A2). The first one is the ProGres data set _,_ which is the UNHCR’s registration system covering all refugees or asylum seekers requiring assistance. The ProGres data set is a live instrument that is continuously updated as new refugees/asylum seekers arrive or existing refugees contact the UNHCR. The data we use were extracted at the end of December 2017. This set of data contains socioeconomic variables (such as household size, marital status, gender, age, country of origin, and region of residence) but no consumption or expenditure data. This data set can be considered as the “census” of refugees.

7
The second set of data, the Targeting data set _,_ is also a census-like data set for refugees living in Chad. The main aims of this census are to fill knowledge gaps on refugee livelihoods and the levels and differences of vulnerability in refugee households, and to categorize refugees into wealth groups for cash, food, and livelihood assistance. In addition to categorizing refugees, the Targeting data set aims to identify factors that can enable self-reliance. This data set is based on a mixed methods approach, including qualitative and quantitative methods. The first step is the use of focus groups with refugee leaders, women’s organizations, and youth associations, to identify the wealth characteristics and key challenges specific to age and gender. Next, a sample survey is carried out across camps to confirm the wealth characteristics that were identified by refugees in the first step. Based on the outcomes of the first two steps, a quantitative survey that is designed to capture wealth characteristics is administered to all the refugee households. <mark>The T</mark> argeting data set <mark>includes all Sudanese, Central African, and Nigerian refugees living in Chad. The data were collected between June 17, 2017 and July 15, 2017 and cover 19 refugee sites and refugees living in nine host villages</mark> **<mark>.</mark>** After the data collection, a statistical model, which takes into account household welfare, is used to classify households into four socioeconomic groups (very poor, poor, average, and better off). For the variables that are relevant for this study, this data set contains demographic variables (household size, gender, age, country of origin, and region of residence), variables for asset and animal ownership, and variables reflecting coping strategies. As for the ProGres data set, the Targeting data set does not contain information on consumption or expenditure. However, it does contain information on wealth.

The last set of data is the Post-Distribution Monitoring (PDM) data set, which is from a sample survey that covers similar themes as the Targeting data set. The PDM data set, which was collected in 2017 by the WFP, aims to provide better understanding of how refugees use food assistance and contains data on consumption and expenditure. The sampling design is a two-stage stratified random sample, where the first stage includes the selection of camps and the second stage the selection of households. Beforehand, the different camps are stratified in three zones: <mark>(i) North East (Ourecassoni, Amnaback, Iridimi Touloum), (ii) Centre-East (Goz Amir, Djabal, Gaga, Teguine, Bredjing, Farchana), and (iii) South (Amboko, Dossey, Gondjé, Belom, Moyo) (Map 1).</mark> In addition, the sampling takes into account the kind of humanitarian assistance that is provided to refugees (in-kind, food voucher, or cash). The survey includes two <mark>consumption aggregates</mark> 8
<mark>measuring monthly total consumption and monthly food consumption, using retrospective questions with varying recall periods depending on the item considered (from seven days to one year). The consumption aggregate is compiled by aggregating the different food and non-food items, including</mark> expenditures on education, health, durable assets, and rent. <mark>In the framework of this study, we consider two welfare indicators from the PDM data set. The first is the household total consumption expenditure per capita per month, and the second is the household food consumption per capita per month.</mark><sup>5</sup> For poverty imputation purposes, three data sets are constructed from the ProGres, Targeting, and PDM data sets. The first, which we refer to as “ProGres 2,” is obtained by appending the ProGres data to the end of the PDM data. As the ProGres and PDM data share only demographic variables, ProGres 2 contains demographic variables for all observations, although only the observations from the PDM data have consumption expenditure. The second set of data constructed, “Targeting 2,” comes from appending the Targeting data to the end of the PDM data. Therefore, Targeting 2 contains demographic variables, asset and animal ownership, and coping strategies variables as well as consumption data. The last set of data, “ProGres Targeting,” is obtained by first merging the ProGres and Targeting data (matching 72% of the observations) and then appending these data to the end of PDM data. This set of data is the more complete data set in terms of variables. <mark>The motivation of the construction of these three sets of data is to check whether the different sources of data as well as different sets of variables generate different poverty figures, consequently determining the set that best predicts poverty. To ensure comparability across the three data sets, we restrict the analysis to 16 of the 19 refugee sites, because the PDM data cover only 16 sites. Consequently,</mark> this study covers only refugees from the Central African Republic and Sudan, and not the Nigerian refugees, and all the estimates provided in the paper are not representative of all the refugees living in the country.

> 5 The aim of the paper is not to measure consumption accurately or estimate nationally representative poverty figures for refugees in Chad. The purpose of the paper is only to test the cross-survey imputation methodology using a sample of refugee data. In this respect, our only concern is that the poverty predictions are close to the poverty rates calculated with the consumption data. Whether our consumption data are accurate or not, this is less relevant for us. We should expect the cross-survey methodology to produce even better results if the quality of the consumption aggregate improves.

9
# **IV. Estimation Results**

As a first step, we check whether our data sets are representative of the same underlying population (Assumption 1) by performing means difference tests across key predictors. Given that the PDM is a subsample of the Targeting or ProGres data sets, we need a test that is suitable for partially overlapping samples. Here we use the same test proposed by Verme at al. (2016, 58) applied to refugee ProGres data in Jordan. Table 1 provides the results. It shows that all the variables are not significantly different in terms of means, indicating that the two samples are representative of the same population.

To evaluate the performance of the welfare estimation model, we consider three models. Model 1 includes demographic and geographic variables (region of residence and country of origin). This is the most parsimonious model and uses the variables that are most readily available in the ProGres data set. Model 2 adds to Model 1 variables related to animal and asset ownership. Model 2 is richer than Model 1, but it is more demanding in terms of the control variables, which may also be less reliable or more likely to be missing in the census data. Model 3 adds to Model 2 variables measuring coping strategies. To test for multicollinearity, Table 2 reports the variance inflation factor (VIF) for the different models. It shows that no variable has a VIF that is over 5, and the mean VIF is smaller than this threshold. We conclude that multicollinearity is not an issue for any of the models considered.

Next, we test the out-of-sample performance and possible overfitting of the three models, using PDM data and the root mean square error (RMSE) and mean absolute error (MAE) as performance functions. To do so, the data set is split into five equal folds. In the first iteration, the first fold is used to test the model, and the rest are used to train the model. In the second iteration, the second fold is used as the testing set, while the rest serve as the training set. This process is repeated until each of the five folds has been used as the testing set. The performance function is obtained as the mean across the five iterations.

For the food consumption aggregate, the three models have similar measures of goodness-of-fit for both indicators. Model 1’s RMSE is 0.55, Models 2 and 3’s RMSE is 0.54. For the MAE, 10
Models 1 and 3 have a value of 0.42, whereas Model 2 has an RMSE of 0.41. When we turn to the overall consumption aggregate, we note a small difference between the three models. The RMSE values range from 0.53 to 0.58, with Model 3 and Model 1 having the smallest and highest RMSE, respectively. The MAE is quite similar across the three models, within a range from 0.39 (Model 3) to 0.41 (Model 2). These results suggest that no model consistently outperforms the other models.

Tables 3 and 4 apply the model to the three data sets described in the data section (ProGres 2, Targeting 2, and ProGres Targeting data), using the normal linear regression model and the Empirical Error Model and three poverty lines: (i) a poverty line of US$1.9 a day (PPP), which represents the international poverty line for extreme poverty (panel A); (ii) the national poverty line, which corresponds to around US$2.6 (World Bank 2013), represented in panel A; and (iii) the national food poverty line of US$1.8 (PPP), shown in Table 3. These three poverty lines are among the set of the arbitrary poverty lines considered in the general simulation above to evaluate the quality of the prediction.<sup>6</sup> With one exception, the predicted poverty rates are not statistically different from the poverty rates obtained from survey consumption data (henceforth, “survey estimates”). For the case of the food and international poverty lines, this is partly due to the large standard errors of the prediction estimates, but these findings hold for the national poverty line where the standard errors of the predicted values are much smaller.<sup>7</sup> Figure 1 repeats the exercise of Table 5 for the ProGres Targeting data and all the poverty lines between the 66th and 99th quantiles of consumption. Panels A and B, respectively, are the normal linear model and empirical error model. The results suggest that Models 1 and 2 predict the poverty rates for different poverty lines well. The predicted poverty rates are within the 95% confidence interval for all the arbitrary poverty lines considered. The predictions are also very similar across the normal and empirical error models. However, Model 3 overestimates poverty, and the predicted poverty rates provided are outside the 95% confidence interval of the survey-based rates for the set of different poverty lines considered. As Model 3 adds variables related to coping

> 6 See also Table A3 in the annex for the full base model.

> 7 The national and international poverty lines are used for illustrative purposes only. As the consumption aggregate used in this paper is not comparable to a full consumption aggregate as usually derived from nationally representative household surveys, the resulting poverty statistics are not comparable to national or international poverty statistics. In our case, these lines are simply alternative thresholds used to test the sensitivity of poverty estimates to different poverty lines.

11
strategies, it might be that households are not accurate in reporting these strategies, for example, by overestimating the frequency of using these strategies to receive more assistance from humanitarian organizations.

Figure 2 shows the predicted welfare rates for the set of different poverty lines for all three models, but this time with a focus on food security. Welfare based on food security is defined in humanitarian settings as the inability to afford the minimum expenditure basket required to purchase a food basket based on basic needs. The minimum expenditure basket is defined by the WFP “as what a household requires in order to meet their essential needs, on a regular or seasonal basis, and its average cost” (WFP 2018). The results are very similar to the overall welfare results displayed in Figure 1. The results indicate that Models 1 and 2 predict the actual welfare rates well based on food security for different poverty lines and are within the 95% confidence interval for all the arbitrary poverty lines considered. The predictions are also very similar for the two different estimation models of error terms, the normal linear model and empirical error model. Again, Model 3 overestimates the poverty rates, as the predicted welfare rates are outside the confidence interval of the survey-based rates.<sup>8</sup> In summary, Figures 1 and 2, corresponding to Models 1 and 2, underestimate welfare for low poverty lines and overestimate for high poverty lines but are within the confidence intervals. Model 3 always overestimates poverty for smaller poverty lines and its predictions are outside the confidence interval. In general, Models 1 and 2 predict poverty and food security poverty well for different arbitrary poverty lines. Based on these results, we conclude that these two models provide fairly accurate aggregate welfare estimations of refugees living in Chad, and that the variables currently available in the ProGres UNHCR registration system can be combined with other survey data to predict the aggregate welfare of refugees efficiently.

The cross-survey imputation literature<sup>9</sup> stresses the importance of selecting a few key predictors, and our results from Model 1, which contains only demographic variables, are in line with this

> 8 To check for possible heterogeneity, we split the sample with respect to country of origin. The results were similar except the larger estimate variances (less precision), which might be due to sample size for refugees from the Central African Republic. 9 See, for example, Dang et al. (2017a); Dang and Lanjouw (2018); Dang and Verme (2019). See also Luca et al. (2018) for a related study on variable selection with linear models.

12
empirical evidence. Previous empirical studies also highlight that adding household assets helps to improve on poverty estimates, and Model 2, which adds asset and animal ownership to Model 1, is consistent with this evidence. However, adding more variables may lead to overfitting, resulting in less accurate welfare estimates. The results of Model 3 could be placed in this context.

# **V. Targeting Performance**

The imputed welfare estimates can be useful in evaluating ex post the inclusion/exclusion errors of the food assistance programs administered by the government and humanitarian organizations during 2016/17. The targeting strategy for food assistance was agreed to and implemented by the <mark>UNHCR, WFP, and the government agency in Chad responsible for refugees, the CNARR.</mark> We perform an analysis to show how accurately the current targeting strategy identifies poor households in terms of inclusion (leakage) and exclusion (undercoverage) errors. Both error types are important but from different perspectives. The inclusion error matters primarily from a budget perspective, as it represents a waste of resources. The exclusion error summarizes the program’s failure to cover households in need.

The current <mark>UNHCR/WFP/CNARR targeting approach relies o</mark> n the Food Consumption Score (FCS) generated by the WFP’s PDM surveys, which is a composite score based on dietary diversity, food consumption frequency, and the relative nutritional importance of different food items. As is the case for any index, the FCS is contingent on the selection of the food group weights as well as the food item thresholds, which are based on inherently subjective choices. Survey-tosurvey methods have been shown to outperform these types of index approaches, whereas the Dang et al. (2017) cross-imputation method has been shown to perform better than the proxy means testing also in refugee contexts (Dang and Verme 2019).<sup>10</sup> In light of the previous findings, we empirically evaluate how the UNHCR/WFP/CNARR targeting strategy performs relative to the targeting method based on predicted consumption and

> 10 On optimal targeting in humanitarian contexts, see also Verme and Gigliarano (2019).

13
relative to the available international evidence. Table 6 shows the undercoverage and leakage rates for the different approaches. The method we propose (panel B) outperforms the targeting method currently used in Chad (panel A) for all the poverty lines except the 25th percentile poverty line. The errors are not low overall, with the UNHCR/WFP/CNARR undercoverage rates ranging from 9% to 32% and the leakage rates from 12% to 36%, and our model-based undercoverage rates from 6% to 40% and the leakage rates from 9% to 41%. However, these methods perform relatively well when compared with international evidence. For example, Skoufias et al. (2001 <mark>) find that the undercoverage and leakage rates for the PROGRESA program in Mexico were 7% and 70%, respectively, for a poverty rate of 25%, a better performance on the undercoverage rate but a much worse performance on the leakage rate compared with the other two programs.</mark> The estimated targeting rates for Chad are also better than the median performance of similar scores for programs across the world (see Table 7). Coady et al. (2004) report an index of targeting performance obtained by dividing the proportion of beneficiaries falling within the target population by the proportion of beneficiaries that would result from a random allocation. For example, if the bottom 40% of the income distribution receives 60% of the funding, the performance indicator is 1.5 (60/40). The higher the indicator is, the greater is the performance of the targeting strategy. Table 6 reports this indicator for the 85 programs considered by Coady et al. (2004) (A), the UNHCR/WFP/CNARR targeting program (B), and our proposed methodology (C). Our methodology outperforms the UNHCR/WFP/CNARR targeting program and the median value of the programs covered by Coady et al. (2004), and the UNHCR/WFP/CNARR program does not perform poorly when compared with the international evidence.

# **VI. Limitations**

The objective of this work was to test how cross-survey imputation methods perform in estimating poverty for refugee populations, using Chad as a case study. Although the results of our crosssurvey imputation exercise show that key demographic variables from ProGres predict well the welfare measure captured in the PDM at the aggregate level, additional work is needed to assess how well this methodology performs in refugee contexts, particularly in poor countries and data scarce environments. To do this, data sets should ideally contain more detailed information on 14
consumption, and they should be matched by individual household, using the IDs available in the ProGres registration data.

Further, the PDM data measure consumption using relatively fewer variables than those in round 4 of the Chadian Household Consumption and Informal Sector Surveys (ECOSIT4). As such, the poverty estimations from the PDM data could be improved. <mark>The poverty estimates used in this paper do not reflect the official poverty estimates monitored by the government and the international community. The interest of this paper was to test a cross-survey methodology, and, for this purpose, we used a subsample of UNHCR refugee data that are not nationally representative of the refugee population in Chad. By contrast, official poverty statistics require national consumption surveys conducted by the national statistical office, with samples that are nationally representative. T</mark> he UNHCR and the World Bank are working closely to improve and increase comparable refugee data to nationals. Thanks to these recent efforts, in 2018/19, refugees were included for the first time in the ECOSIT4. When these data become available, it will be possible to run a similar analysis to assess how cross-survey imputation fares using nationally representative consumption measures for poverty, and to understand how cross-survey imputation can predict household poverty outcomes for refugees and host populations alike with comparable data.

Although the work presented in this paper remains a valid experiment for cross-survey imputations, the data did not cover the entirety of refugees in Chad, including some refugees who live outside camps. As the latter live in different environments, predicting their welfare may require different sets of variables. And measuring consumption among refugees who rely on a combination of handouts and informal incomes is a relatively new science. Existing survey instruments may need to be adapted, and the meanings of concepts such as utility and capabilities among refugees need to be reconsidered.

# **VII. Conclusion**

UN General Assembly Sustainable Development Goal 1— _End poverty in all its forms_ by 2030— explicitly pledges that “no one will be left behind.” Tracking the progress made toward this objective requires the availability of high-quality household consumption surveys. However, the 15
majority of countries across the world, especially developing countries, face challenges in collecting poverty data. High-quality consumption surveys that are comparable for forcibly displaced persons and their hosts are and will remain in limited supply, given the cost and challenges associated with these types of surveys. In the meantime, cross-survey imputation methods can provide a second-best alternative that can potentially save time and money.

<mark>This study combined</mark> survey and census-type data on refugees to estimate welfare for <mark>refugees in Chad. W</mark> e showed how different sets of variables as well as different sources of data fare in the identification of poo <mark>r households, in particular, how well the set of variables available in the ProGres database can predict poverty. In a second step, the paper estimated the accuracy of the current U</mark> NHCR/WFP/CNARR <mark>targeting strategy and compared it with the targeting strategy based on imputed consumption in the light of international evidence.</mark> The results suggest that the set of variables available in ProGres accurately predicts the welfare rates for different poverty lines. Adding variables related to asset and animal ownership provides predictions that are very close to the ones with only the variables available in the ProGres data set. These results are especially promising, as the UNHCR ProGres data are available in most refugee locations where the UNHCR runs the registration system, and thus these methods are replicable in many settings of forcibly displaced persons.

The <mark>current targeting strategy that is used for food, livelihoods, and cash-based assistance, despite its simplicity, is rather accurate when compared with the existing international evidence. Th</mark> e targeting errors resulting from the current <mark>UNHCR/WFP/CNARR targeting strategy for a poverty rate of 25% are in the same error range as other targeting methods around the world, as reported in Coady et al. (2004). We also showed that the existing targeting method can be improved by imputing consumption using the methodology proposed in this paper.</mark> 16