The results in Table 4.5a have provided evidence that the bias adjustment methods employed in the HFPS in the four countries have been effective at reducing the bias present in the interviewed sample. The benefits can be further illustrated by examining not just the point estimates for the considered characteristics but also the confidence intervals. Figures 4.2 to 4.8 plot the point estimates and 95 percent confidence intervals for selected characteristics for the full LSMS-ISA (F2F) sample, HFPS sample with adjusted weights, and HFPS sample with unadjusted weights. In the figures, all estimates are standardized by subtracting the F2F survey mean (so the F2F mean will always be zero) in order to allow comparison across indicators. For household size and head age, the estimates were further standardized by dividing by the F2F mean. Thus, the confidence intervals represent the percent deviation from the mean for these two variables.

Looking through the figures, it is clear that the adjustments for bias result in substantially improved alignment with the LSMS-ISA survey point estimates and confidence intervals. Although the level of reduction in the bias achieved varies across indicators, the adjustment for nearly all indicators does result in a shift of HFPS estimates towards the LSMS-ISA. For many indicators, the reduction is the bias from the adjustments is substantial, for example literacy in Figure 4.2. As expected, the bias adjustment was especially effective for the indicators that were included in the bias adjustments in each country (listed in Table 3.2). However, for other indicators, the reduction in bias is more modest. For example, the housing characteristics in Figure 4.5 show substantial bias without the adjustment to the weights and the bias is significantly reduced after applying the adjustments, but there still remains a gap between the point estimates for the F2F and HFPS samples (though not statistically significant).

Comparing the results for the four countries, there is remarkable consistency in the direction of the bias across the indicators, though the magnitude of the bias varies. The bias is always towards households that are better off in terms of consumption expenditures and general living standards. This is especially evident in Figure 4.4 for consumption expenditure quintiles where the bias follows a linear pattern with the largest bias among the poorest and richest households and in opposing directions (negatively biased for the poorest and positively biased for the richest). In this case the bias adjustment is quite effective in all four countries (and was included in the adjustment model).

While Figures 4.2 to 4.8 illustrate the effectiveness of bias adjustments for dichotomous variables, Figure 4.9 goes a step further to show how effective the adjustments are for continuous welfare measures: consumption expenditures for Ethiopia, Nigeria and Uganda and wealth index for Malawi. Comparing the distribution for the full LSMS-ISA sample with that of the HFPS without the bias adjustments, the modest bias can be seen in all four countries, particularly in Ethiopia and Malawi. However, the HFPS sample distribution with the bias weighting adjustments aligns much more closely with the LSMS-ISA distribution, demonstrating the effectiveness of the bias corrections across the distribution of continuous measures.

Although the bias adjustments have broadly been shown to substantially reduce bias in the estimated mean of the examined indicators, one concern is that the adjustments to weights will result in a higher variance in the weights and thus larger standard errors. Large differences between the LSMS-ISA and HFPS interviewed sample could result in extreme outlier weights following the weighting adjustments. Larger standard errors would compromise the precision of the estimates and, if the standard errors are too large, will limit the usefulness of the data. In order to counteract this potential, an additional step (outlined in Section 3.1.2) is implemented in the response propensity correction factors by sorting into deciles according to the correction factor value and applying the mean correction factor within each decile. The impact of the adjustments on standard errors can be observed in Figures 4.2 to 4.8 by comparing the width of the confidence intervals of the HFPS sample estimates with and without the bias weighting 32