# <mark>3.1.3 Method of Analysis</mark>

The analysis undertaken in this study involves two steps. First, the magnitude and different sources of bias introduced in the phone survey selection and interview process are identified. Several subsamples are compared, reflecting the eligibility, selection, and interview process for the HFPS in the four countries. To measure the bias, pairwise comparisons of weighted means across a wide variety of household characteristics is carried out. In order to assess the extent of **coverage bias** , differences between the full F2F survey sample and the _eligible_ sample for the HFPS (i.e. households with contact information) are examined. To measure **sample representativeness** in Nigeria, the profile of the selected sample is compared with the frame of eligible households. Lastly, to estimate **nonresponse bias** , the sample of households successfully contacted and those successfully interviewed is compared with the overall HFPS sample. Comparing the profile of characteristics for these samples will provide a more refined impression of the bias introduced throughout the course of the phone survey process. This comparative analysis will also reveal at what stage in the selection process the bias is the most substantial.

After conducting the comprehensive assessment of bias, an assessment of the reweighting adjustments to counteract bias is conducted to see how effective these methods were at reducing bias. Weighted estimates for the diverse set of characteristics examined in the first analysis is presented for the full representative F2F survey as well as the HFPS. The F2F survey characteristics serve as the benchmark and are compared with the profile of characteristics obtained from the HFPS sample applying the weights that include the bias adjustments (as described in Section 3.1.3). Comparing these two sets of weighted characteristics will show how much bias remains in the sample after the correction. However, the relative effectiveness of the bias adjustments is further analyzed by computing a second set of HFPS weights which _exclude the bias adjustments_ . The process of calculating these “unadjusted” weights follows exactly the same steps as the “adjusted” weights and only excludes the bias adjustment. The unadjusted weights serve to simulate the extent of bias that would have been present without the bias adjustments. By comparing the relative deviations from the F2F sample profile, this will provide an indication of how effective the bias adjustment techniques employed were in reducing the extent of bias in the HFPS sample. Finally, the advantages of drawing phone survey samples from representative F2F surveys is highlighted based on the performance shown using adjusted weights when compared to unadjusted weights.

All the pair-wise comparisons of characteristic means in the analysis below are conducted following the same approach. All means are weighted using either the LSMS-ISA weights or weights calculated from the HFPS and the sampling properties of each survey are accounted for (clustering, stratification, etc.). An adjusted Wald test is used to determine if differences in the means between the different subsamples are significant.

10