In line with the findings and recommendations from Corral et al. (2022), gradient boosting methods appear to outperform the traditional small area methods recommended for off-census years. The unit-context and Fay-Herriot models are direct competitors to the machine-learning models, which are often motivated by a lack of up-to-date census data. Also aligned to Corral et al. (2022), area-level models outperform unit-context models in terms of bias and MSE.

Now that we have established that in the context of the Mexican data used and with covariates derived from census data, gradient boosting methods can perform just as well as CensusEB methods our attention is turned to gradient boosting’s performance under geo-referenced data – Gradient Boosting (GIS-MUN).<sup>31</sup> The gradient boosting model with GIS-based covariates is our closest approximation to the standard implementation of machine learning in the context of poverty mapping, the method is contrasted to an application of an area-level model with similar data – Area-level (GIS-MUN).<sup>32</sup> The median MSE for the gradient boosting model with GIS-based covariates is 0.008 (with a root MSE of about 0.09), meaning that the median municipality’s error is roughly nine percentage points. The area-level model using geo-referenced covariates achieves a median MSE of 0.01.

The bias associated with poverty estimates is especially important due to the policy relevance of understanding absolute living standards. In panel (b) of Figure 2, we thus present box plots for the municipality-level bias estimates associated with each model (see Eq. [26] for the bias-variance decomposition of the MSE). The median bias for all models is approximately zero, with the possible exception of the unit-context model and the area-level model estimated at the PSU level.<sup>33</sup> Perhaps the more interesting result is that the bias estimates from the gradient boosting model with georeferenced covariates exhibits considerable variation, with minimum and maximum biases reaching -0.43 and 0.43, respectively. That is, the standard implementation with remotely-sensed covariates is severely biased for some municipalities, even though its median bias is approximately zero. All other models, including the competing unit-context and area-level models, exhibit much less variation.

The performance of machine learning appears to depend heavily on the covariates used in the model, with census-based covariates considerably improving model performance. Another way to see that the census-based covariates have greater predictive power than the remotely-sensed covariates is by looking at feature importance. One popular measure of feature importance summarizes the gain (i.e., reduction of the loss function) associated with any given feature. That is, for a given run of the model, feature importance captures the average gain of a given feature, where the gains are averaged across all trees in the model.<sup>34</sup> Figure 3 plots the 20 covariates with the highest feature importance

> 31Note that the underlying model for CensusEB estimates relies on household level microdata. Consequently, the data requirements for CensusEB are even higher.

> 32A notable aspect of Fay-Herriot and Gradient Boosting models is their relative ease of application and the fact that these usually take considerably less time in implementing as opposed to CensusEB and unit-context models.

> 33The same issue for unit-context models is highlighted by Corral et al. (2021a) and Corral et al. (2022).

> 34In addition, the average gains for a given run of the model are normalized by dividing each by the sum of all average gains for that run.

22