# 6. Conclusions

In this paper, we evaluate different alternative methods to identify poor villages in 10 Malawian districts. This is a challenging prediction exercise because villages are highly geographically disaggregated. The results show that a two-step approach utilizing a hypothetical partial registry from 450 villages performs vastly better than the PMT, geospatial prediction based solely on the household survey, or the Meta relative wealth index. The main measure used to identify poor villages is the mean predicted per capita consumption of the bottom half of households in each village, but key results hold when using the mean predicted per capita consumption of all village households as the village welfare measure.

Implementing the partial registry method requires nationally representative survey data, publicly available geospatial indicators, and the collection of a partial registry containing a subset of household characteristics found in the survey data. Several similar household surveys that collect information on welfare proxies have been fielded with the support of the World Bank through the Survey of Well-Being with Instant and Frequent Tracking (SWIFT) program, including in Malawi. Although none have surveyed the full population of households in selected villages, it is quite standard for household surveys to list all surveys in sampled enumeration areas, and we estimate that the cost of collecting approximately 40 variables from all households in approximately 500 villages could be in the ballpark of $24,000 to $73,000. This is a worthwhile investment to greatly boost the accuracy of village welfare measures constructed using geospatial data. Some countries also field periodic community surveys, which could potentially be tweaked to collect partial registries.

This paper also demonstrates the efficacy of applying gradient boosting models in settings with household-level predictors, when sufficient data are available. In particular, using training data from the full sample of households, rather than only the 10 districts of interest, substantially increases predictive performance. Using post-lasso models instead of gradient boosting in the geospatial model, the second step of the partial registry method, only slightly affects the accuracy of the predictions. In contrast, using post-lasso models instead of extreme gradient boosting in the census model, used to impute per capita consumption into the simulated partial registry, greatly reduces the predictive power of the geospatial model. This large reduction in predictive power occurs both when using gradient boosting and post-LASSO for the geospatial model. This suggests that the census data may contain outliers, which introduce more noise into the partial registry predictions when using a linear model of log per capita consumption than when using gradient boosting.

The relatively poor performance of the PMT scores derived from the UBR data is a puzzle. The UBR PMT scores performed a bit better when the benchmark measure of welfare was constructed using data only from the 10 Malawian districts. Even so, the UBR PMT scores do not appear to be consistent with welfare predicted using the census data collected from these districts. Partly this may be due to the UBR data being taken from the initial phase of data collection, although it is also possible that the purpose of the partial registry may have led to measurement error. It would be useful to do these types of evaluations with further rounds of the UBR, even if compared against old census data, to see if later rounds of the UBR produce predictions that are more consistent with the census.

24