factor is defined as the ratio of the estimated poverty rate obtained from the survey to the populationweighted mean poverty rate of the small area estimates, for each super-area.

This leaves open the question of how to adjust the mean squared error estimates while benchmarking. Ideally, to obtain accurate estimates of the mean squared error, benchmarking would be performed by the small area estimation package within each bootstrap replication. Unfortunately, this option is not currently implemented in available software packages. As a second-best solution, we scale the mean squared error by the square of the same scaling factor that is applied to the point estimates, which leaves the coefficient of variation unchanged by the benchmarking process. Because the point estimates are slightly underestimated in Tanzania and Sri Lanka, this procedure increases the mean squared error and improves coverage rates. However, as a robustness check, we also report results for estimates in which only the point estimates are benchmarked, and in which neither the point estimates nor the mean squared errors were benchmarked.

# H. Criteria for evaluating estimates

The synthetic sample, auxiliary data, and methodology can be used to generate small area estimates of headcount poverty and their mean squared error. But how exactly does one evaluate the estimates produced by different small area estimation methods? We examine seven summary statistics, averaged across areas. We give each area equal weight in the evaluation to better reflect the goal of obtaining accurate estimates for each target area rather than estimates that can be aggregated to higher levels. The six summary statistics are divided into four buckets as follows:

- **Mean Poverty Rate and Average Relative Bias.** The former represents the simple average of the predicted area headcount rates across areas, prior to benchmarking. For the direct survey estimates, the mean poverty rate is the unweighted average poverty rate across municipalities. The mean of the small area estimates can differ from this unweighted average, due to bias caused by either violations of the normality assumptions or a non-representative sample. Substantial bias would necessitate a greater reliance on benchmarking, which would in turn make the imperfect adjustment for mean squared errors more problematic. Therefore, estimators with mean poverty rates closer to those of the direct estimates are, ceteris paribus,

preferred. Meanwhile, the average relative bias equals the average, across areas of the ratio of the difference between the true and estimated poverty rates to the true poverty rate. ARB =<sup>1</sup> NNaa �<sup>Θ�aa</sup> Θ<sup>−Θaa∗</sup> Where Na is the number of areas, Θ<sup>�</sup> is the estimated poverty rate for area a, and aa aa<sup>∗</sup> Θ is the true NN aa=1 poverty rate calculated from the census. Like the average poverty rate, it is a measure of the aa aa<sup>∗</sup> systematic bias in the estimates rather than accuracy. It differs from the average poverty rate, however, by reflecting any systematic bias at the area level after the benchmarking procedure.

- **Average Mean Squared Error (MSE) and Average Coefficient of Variation (CV).** These measure the degree of uncertainty associated with the point estimates. Estimation of the MSE for the

18