# **4 Results**

## **4.1 Response rates, randomization check and sample characteristics**

Panel A of Table 1 shows that only 12% of the 5,229 baseline respondents completed either or both of the follow-up surveys. Attrition in the medium-term sample affected each arm equally. A similar finding applies to the short-term sample, with the only difference that in this case the explicit (Treatment 2) arm exhibits a lower response rate due to a technical constraint triggered by a policy change in Facebook Messenger while we were conducting the study.<sup>23</sup> The low follow-up response rates naturally affect the study’s statistical power, especially for the medium-term survey, where the magnitude of the effects are generally smaller.<sup>24</sup> Low response rates could also jeopardize the external validity of the study if individuals with certain characteristics have a higher propensity to complete the study. Annex Table A1 compares pre-intervention characteristics of those respondents who filled the follow-up surveys with those who only completed the baseline and never continued. We find evidence of self-selection into study completion with respect to gender-related outcomes, though the different samples are generally similar across most socio-demographic characteristics.<sup>25</sup> Since respondents in the final samples had ex-ante more progressive views, we expect our estimates on knowledge and attitudes to represent a lower-bound of the campaign impact, because of the lower margins for improvements in the higher baseline values.

To assess whether attrition invalidated the randomization strategy, which may pose a threat to internal validity, Annex Tables A2 and A3 show differences in sample means between experimental groups of outcomes and covariates measured at baseline, only for those re23The new policy, which has since been modified again, prevented chatbots from sending automated follow-up messages. It went into effect one week after we began our study, which forced us to compress the timing and send the follow-up survey exactly one week after the study began, before the new policy prevented the follow-up from being sent. This disproportionately affected the documentary watchers, as they had more episodes to watch. While many were contacted manually later, they were still less likely to respond.

> 24The study’s initial power calculations estimated a two-sided test with power of 0.8, alpha of 0.05 and no intra-cluster correlation for the question “Do you think a husband is justified in hitting or beating his wife if he suspects her of being unfaithful” among 18-24 individuals living in urban India (NFHS-4). To detect a six-percentage point increase in this outcome, each treatment arm required around 500 observations. However, each follow up survey had approximately 200 observations per arm.

> 25The two panel subsamples tend to exhibit overall more progressive attitudes than the only-baseline subsample, and the differences are generally statistically significant. For instance, panel subsamples are less likely to justify domestic violence and more willing to report it. This is a consequence of this study trying to replicate the real world, where media consumption is an individual choice and tends to be biased. At the same time, it also suggests that program content matters and raises the potential concern that even the implicit format (a humorous fake reality TV show) could fail to attenuate self-selection into viewership by reaching and engaging individuals with different interests. On the other hand, the different subsamples are generally similar across socio-demographic characteristics, with the exception of female respondents, respondents with sisters and respondents with high media consumption who are all more likely to complete the two follow-ups.

13