crime surveys conducted in Germany and found that interviewers were accountable for 77 percent of the total clustered variance with interview areas only accounting for 23 percent.

# _Overview of research_

These findings, which are typically not accounted for in survey designs, mean that surveys are likely underpowered. This paper sets out to quantify interviewer design effects in the context of a developing country household survey conducted with CAPI. To do this, we attempt to isolate the portion of the variance attributable to interviewers from the portion attributable to interview areas. We also control for interviewer – respondent interaction effects that may be associated with the gender of interviewer and respondents. We make recommendations for survey design and data analysis to mitigate the effect of interviewer design effects.

The paper proceeds as follows. Section 2 describes the data, including sampling and the variables selected for this study. Section 3 describes our analytical approach, while section 4 presents and discusses the results. Finally, section 5 presents a brief conclusion and recommendations.

# **2. Data description**

The data used in this study is from the Agricultural Production Survey (APS), a pilot survey conducted by the Central Bureau of Statistics of Sudan in collaboration with the World Bank in March-April 2018. The survey was conducted in Kassala State of Sudan. The APS questionnaire was designed largely based on the World Bank’s Living Standards Measurement StudyIntegrated Surveys on Agriculture (LSMS-ISA). The questionnaire included four modules: (1) the household module, (2) the agricultural planting module, (3) the agricultural harvest module, and (4) the livestock module. For this study, we only use the household module because this was the only module completed for all households and therefore provides the largest sample size. The data was collected with tablets using the Survey Solutions CAPI program.

# _Sampling_

The sample was drawn using a two-stage cluster design. First, the simple random sample size was calculated with the standard parameters of a 5 percent error margin, a 95 percent confidence level and a key parameter expected value of 0.5.<sup>2</sup> Next the area cluster design effect was calculated to account for interview area clustering using formula 1 above with cluster size _c_ equal to 10 households and ICC _p_ equal to 0.15.<sup>3</sup> The result was a design effect of 2.35, which was multiplied by the SRS sample size (of 384) to yield a total sample size of 903 households.<sup>4</sup> In the first stage of sampling, Popular Administrative Units (PAUs) were randomly selected prior to the survey. Then each day within each cluster (PAU), the survey team carried out a listing

> 2 The sample size is determined using this formula: nsrs = t<sup>2</sup> × pm(<sup>2</sup> 1−p) , where _nsrs_ is the sample size with SRS sampling, _t_ is the confidence level, p is the estimated value for the key parameter and m is the margin of error.  Note that 0.5 is the most conservative value for _p_ in that it results in the largest sample size.

> 3 This ICC was estimated based on values used in other agricultural studies in other countries.

> 4 Three replacement clusters were also included in the survey. These were mistakenly included in data collection resulting in a larger than necessary sample size of 93 clusters with 930 respondents.

4