model-based estimators are achieved at the expense of design bias. As an additional check, we add simulations where _L_ = 500 samples are taken, each consisting of a 1% SRS without replacement in every PSU within the fixed census population. Aggregate results for this scenario are presented in column 4 of Table 2 and box plots for MSE and bias are presented in appendix Figures 29 and 30, respectively. The first thing to note in these figures is there are fewer extreme outliers when compared to results from the two-stage sample scenario in Figure 31 in the appendix. Despite fewer outliers, the results under the additional sampling scenario mimic those from two-stage samples. However, under the two-stage sampling used here, direct estimates for most municipalities are not available across all 500 samples (see subsection 4.1). Consequently, direct estimates are not included under the remaining figures that discuss results from the two-stage samples.

Under the simulation experiment with two-stage sampling, the empirical MSE of _ELLc_ estimates, where the location effect has been specified at the PSU level, appear to have a tighter spread than direct estimates (Fig. 14). Consequently, though ELL appears to perform relatively well, relative to direct estimates, the number of ELL outliers with a high MSE is considerable and _ELLc_ performs worse than all other small area approaches. This result was not expected, as the FGT0 results for _UC − ELLc_ appear to perform better than the traditional _ELLc_ which has a considerably better model fit.<sup>44</sup> Nevertheless, _UC − ELLc_ does very poorly on mean welfare where it has an MSE that is as large as that of direct estimates and is also considerably biased (see appendix Table 7). These results also hold under the simulation experiment with the 1% SRS by PSU samples.

As expected, _CEBa_ estimates show a considerably tighter MSE spread than direct estimates, but still with outliers. Another interesting finding is that under CensusEB but with random effects specified at the PSU level ( _CEBc_ ), the empirical MSE is tighter than that of the direct estimates and also displays better properties than _ELLc_ as seen in Table 2. However, given the discussion in the model-based validation and the results shown here, the results with the random effects at the municipality level are preferred over the ones where the effect is specified at the PSU level. Perhaps the most surprising result is the low MSE of the _UC − CEBa_ method with only contextual variables and the other _UC − CEB_ variants, as proposed by Masaki et al. (2020). The results display a tight spread and the results from Table 2 corroborate the finding. Nevertheless, beyond _UC_ models being an alternative if contemporaneous census data is not available, the method presents a couple of advantages over traditional Fay Herriot (FH) area level models under sampling scenarios that follow a two-stage design like the one considered here. First, as noted toward the end of section 4.1, the majority of municipalities are represented by 1 PSU and thus have quite small sample sizes which makes the likelihood of direct FGT0 estimates being equal to 0 or 1 much higher. Under these cases, the method from Nguyen (2012) is a valid alternative to FH because FH is not applicable in these municipalities with a sampling variance of 0. An additional advantage is that the model can be used for multiple indicators whereas the FH requires a model for each indicator considered.

> 44Note that under the MI inspired bootstrap of ELL, a very poor model fit will be heavily penalized and will likely yield noise estimates for _UC − ELLc_ that fare worse than direct estimates in most applications.

28