mapping methods to alleviate poverty in the context of the Mexican data.

The results of our simulations can be summarized as follows. First, we find that the magnitude of the downward bias of the _R_<sup>2</sup> based on direct estimates is large, with the true _R_<sup>2</sup> being around 35 to 50 percent higher depending on the model considered. We further find that this bias has important implications for model selection in that the direct _R_<sup>2</sup> incorrectly identifies the appropriate level of estimation in the vast majority of our simulations. Second, when assessing model performance based on the true poverty rates using the mean squared error, we find that our closest approximation to the standard machine-learning implementation performs poorly relative to several benchmark models. We find that these performance issues are largely due to the limited predictive power of remotely-sensed covariates relative to census-based covariates. Finally, our targeting simulations show that none of our machine-learning implementations outperform more traditional poverty mapping methods that are feasible with the same data.

Our paper builds on the work of Corral et al. (2021a) and Corral et al. (2022), who similarly used the Mexican Intercensal Survey to examine the performance of several poverty mapping methods. Corral et al. (2021a) focused exclusively on traditional poverty mapping approaches and did not consider the performance of machine-learning methods. While Corral et al. (2022) consider the performance of machine-learning approaches, their implementation only uses census-based covariates and thus does not examine the performance of the standard implementation that relies on remotely-sensed covariates. Importantly, we are not the first paper to evaluate the performance of machine-learning methods relative to census data. While Yeh et al. (2020) and Chi et al. (2022) validate their models relative to censuses, both papers use the _R_<sup>2</sup> as a performance metric and focus on wealth estimates rather than poverty. Finally, van der Weide et al. (2022) examine how well machine-learning methods using remotely-sensed data can reproduce poverty maps estimated using census data. They do not, however, compare their estimates to a ground truth.

This paper then represents the first attempt to rigorously assess the performance of modern poverty mapping methods relative to true poverty rates. In what follows, Section 2 discusses the data we use for our experiments and Section 3 discusses the various methods we examine, including both machine-learning methods and some traditional poverty mapping methods that we use as performance benchmarks. Section 4 then considers the issue of model validation where we critically assess the _R_<sup>2</sup> as a validation metric and then propose some alternative metrics. Finally, Section 5 presents the results of our experiments and Section 6 provides some concluding remarks.

# **2 Data**

The Mexican Intercensal Survey was carried out by the Mexican National Institute of Statistics and Geography (INEGI). The sample consists of 5.9 million households and is representative at the national, state (32 states), and municipality level (2,457 municipalities). It is also representative for localities with populations of 50,000 or more inhabitants. The administered questionnaire gathered 4