reported was 0.171 or about half of the high ICC estimated in this research. Brunton-Smith _et al._ (2016) found interviewer design effects between 2.5 and 3.3 using a similar number of interviews per interviewer as the survey we use here.

# _Interviewer characteristics_

Annex 2 shows the results using Specification IV. This specification adds to Specification II interviewer characteristics for the three dependent variables with the highest ICCs in Table 3 above. These results are presented only in the annexes because they are based on a limited sample since data from all interviewers was not available. Even if data for all 20 interviewers were available, this would not be a large enough sample from which to draw statistically robust conclusions. These results therefore should be interpreted with caution and be considered only as suggestive rather than empirical. The inclusion of different interviewer characteristics reduces ICCs by between 6 and 14 percent with a mean of 10 percent when compared to ICCs calculated using the same sample and model but without interviewer characteristic covariates. The most significant interviewer characteristics are having lived in Kassala State (14.1 percent lower ICCs), having a university degree or higher (13.4 percent lower) and experience working for CBS before (13.1 percent lower).

# **5. Conclusions and recommendations**

Researchers have long been aware of the need to consider geographic clustering of responses in multi-stage cluster designs. Estimated design effects are used to calculate the sample size increase needed to account for increased variance due to within-cluster homogeneity of responses. No equivalent practice, however, is used to account for potential non-random distribution in measurement errors that results in homogeneity in data collected by the same interviewer. Using data from a pilot survey in Sudan, we find that clustering of responses by interviewers can in fact lead to larger design effects than area clustering. This finding is consistent with the findings of Schnell and Kreuter in Germany (2005).

This paper provides evidence that simpler questions lead to smaller interviewer design effects. This finding reinforces the intuition that time invested in designing, testing and refining questions to make them as simple as possible is likely to yield data quality improvements.

In general, good survey implementation practices can contribute to lower interviewer design effects. These include recruiting qualified interviewers; budgeting adequate time for questionnaire design, review and programming for tablets; conducting a high-quality training using a close to finalized questionnaire; budgeting adequate time for piloting and finalizing the questionnaire; and managing survey implementation well, including early identification of reoccurring interviewer-level issues that might affect data quality.

Another recommendation emerging from these findings is that interviewer identifiers should always be included in all survey data. Interviewer identifications in data allow for multi-way cluster robust standard errors by both area cluster and interviewer (Cameron _et al._ , 2011). Given the evidence of interviewer design effects in survey data collected in developed countries and 10