for them. More respondents in the list sensitive group reported zero than in the non-sensitive group. This implies that the mean in the list sensitive group is likely to be downwards biased, implying the same for the difference-in-means estimate and the resulting IPV prevalence.<sup>30</sup> Further, the one Rwanda question that did not fail the no design effects test showed the greatest difference in prevalence between the list and other methods. It therefore seems plausible that, if biased, the three Rwandan list experiments that failed the no design effects test produced downwards biased prevalence estimates. The associated test p-values and proportions are shown in Tables A11 and A12.

There is no explicit test for the second ‘no liars’ assumption underpinning the list experiment. However, the most obvious threat to this assumption is when a respondent’s anonymity for their answer to the sensitive statement is compromised. This ‘ceiling effect’ exists when a respondent’s true answer is affirmative for all of the statements, thus their affirmative response to the sensitive statement will no longer be hidden. However, the proportion of respondents answering with the maximum number of items (three and four for the non-sensitive and sensitive list groups, respectively) is within the range of other studies.<sup>31</sup> Further, the presence of a ceiling effect likely indicates downwards biased estimates because other respondents would be likely to have understated their true number of 4 affirmative answers in order to conceal their affirmative sensitive answer (Blair and Imai, 2012).

# **4.3.2 Checks on overall approach**

The two key assumptions underpinning the unbiasedness of the list experiment estimator have parallels for the experiment as a whole. While there is no direct test for these assumptions holding for the ACASI and face-to-face methods, there are some indicators of whether or not they were violated.

First, we checked balance on survey questions asked of all respondents after the experimental violence questions. If respondents were differentially affected by one survey method, perhaps indicating a design effect, we may observe them answering subsequent survey questions differently. We found very few statistically significant differences in respondent answers despite the many tests for significance. None of these differences are materially significant.

Second, we analyzed the main list method prevalence regressions with enumerator and controls selected using post-double-selection LASSO (Belloni, Chernozhukov and Hansen, 2014). We found no notable differences in coefficients or significance. These results are shown in Tables A13 to A15.

Third, although a general ‘no liars’ assumption is untestable, we compared refusal rates across methods. This could indicate the relative risk of misreporting for each method. The refusal rate with ACASI, where all one has to do to skip a question is touch a button, may be a lower-bound on the share of people who might have lied if they were interviewed face-to-face and would be uncomfortable asking an

> 30One study by Flavin and Keane (2009) evaluates whether the design of the non-sensitive list affects responses. This study found that asking the non-sensitive group directly about the non-sensitive items resulted in a higher mean number of items than when the non-sensitive questions were asked using the traditional ‘how many of the following items’ approach. As a result, the direct approach to asking the non-sensitive questions resulted in downwards-biased prevalence estimates, though it should be noted that their ‘non-sensitive’ items were somewhat sensitive. Further, a recent paper by Chuang et al. (2018) found that having relatively sensitive ‘non-sensitive’ statements on the list appears to mask the sensitive item of interest, and results in lower variance and higher prevalence estimates. In Rwanda and Nigeria, we used relatively innocuous non-sensitive list items, which, if results from the Chuang et al. measurement experiment on risky sexual behavior hold, then it again seems plausible that these list experiment results may be downwards biased.

> 31The proportion of respondents in the non-sensitive list group reporting three items as true is never greater than 15.3 percent (for the women’s physical violence question in Rwanda), and the proportion of respondents in the sensitive list group reporting the full number of items is never greater than 5.4 percent (for the women’s physical violence question in Nigeria).

19