true _R_<sup>2</sup> to a constrained version of the true _R_<sup>2</sup> that fixes the intercept and coefficient to zero and one, respectively. By constraining the _R_<sup>2</sup> in this manner, we remove its ability to adjust for systematic differences between the poverty estimates and the true poverty rates. Panel (b) of Figure 1 presents the results and we see that, as expected, the unconstrained _R_<sup>2</sup> overstates the model’s performance, albeit only to a small degree. We nevertheless find that even these small differences can lead to model selection issues. In particular, the constrained _R_<sup>2</sup> identifies the PSU level as the appropriate level of estimation for 355 of the 500 samples whereas the unconstrained version, as mentioned, selects the PSU level for 437 out of the 500 samples. The commonly used unconstrained _R_<sup>2</sup> then selects the incorrect model for 82 of the 500 samples.

Given the shortcomings of the _R_<sup>2</sup> , we have argued that the empirical MSE of the poverty estimate at the area level represents a better metric for validating the performance of a particular set of estimates. Panel (a) of Figure 2 thus presents our empirical MSE results for several alternative models. The first three models apply gradient boosting at the municipality level using three alternative covariate sets: geo-reference covariates (“GIS-MUN”), census-based covariates (“CEN-MUN”), and then a model with all available covariates (“ALL-MUN”). The fourth model applies gradient boosting at the PSU level and only uses census-based covariates (“CEN-PSU”).<sup>29</sup> The final three models are all traditional methods that we use to benchmark the performance of gradient boosting. We present unit-level (CensusEB), unit-context, and area level models. We consider three alternative implementations of the area-level model that, much like for gradient boosting, vary the level of analysis and the covariates used. In particular, the empirical best (CensusEB) model is often considered the “gold standard” of small-area estimation and corresponds to an ideal scenario where up-to-date, micro-level census data are available. For these traditional models, all available covariates enter the models linearly (i.e., we do not include interaction or quadratic terms).

One advantage of the MSE is that it can be calculated for each geographical entity. As such, each box plot in panel (a) of Figure 2 summarizes the distribution of municipality-level MSE estimates rather than the aggregate MSE for each sample (i.e., the results are based on Eq. [26]). It is useful to first consider how gradient boosting performs relative to the traditional models when using the same census-based covariates. As is evident in the results from panel (a) and (b) of Figure 2 the gradient boosting methods using census aggregate data – Gradient boosting (CEN-PSU) and Gradient boosting (CEN-MUN) – perform better than unit-context and Fay-Herriot models, despite using similar data, and approximate the performance of unit-level (CensusEB) estimates. The median MSE for the CensusEB model is 0.003 while that of gradient boosting methods is 0.002 and both methods have similar interquartile ranges. The results suggest that gradient boosting methods can approximate CensusEB unit-level methods and will likely outperform unit-context and Fay-Herriot methods by a considerable margin, which is quite encouraging for cases where estimates are needed when the survey and census data are not contemporaneous.<sup>30</sup>

> 29Recall that the geo-referenced covariates are only available at the municipality level.

> 30The unit-context models include covariates at the PSU and municipality level. The location effects are specified at the municipality level.

20