metadata: authors: 0: author: Finn,Arden Jeremy
1: author: Anderson,Jessica Marie
2: author: Aghajanian,Alia Jane
count: West Bank and Gaza
docna: 0: docna: A Tale of Two Surveys : Comparing the
            Outcomes of an In-Person and Web-Based Survey of Mental
            Health in the West Bank and Gaza
docty: Policy Research Working Paper
theme: Data production, accessibility and use,Gender,Human Development and Gender,Data Development and Capacity Building,Economic Policy,Rural Development,Fiscal Policy,Institutional strengthening and capacity building,Public Sector Management,Fiscal sustainability,Urban and Rural Development,Geospatial Services
abstracts: cdata!: Internet surveys may never replace
            in-person surveys as a gold standard, but they remain
            important tools for rapid, remote, and low-cost data
            collection. The West Bank and Gaza Poverty and Equity team
            had a unique opportunity to compare a Facebook survey with
            an in-person survey covering conflict exposure and
            potentially associated socioeconomic and mental health
            outcomes over a similar time period. It is reasonable to
            expect that the estimates from internet surveys and
            in-person surveys would differ. In this case, the Facebook
            survey estimates more severe outcomes (e.g., higher exposure
            to conflict and worse mental health) than its in-person
            counterpart for most topics and populations.  Multiple
            mechanisms may have contributed to this difference in
            estimates, including overrepresentation in the Facebook
            sample of respondents who were interested in the survey
            topics, reduced sensitivity bias in the context of a
            self-administered online questionnaire, and reporting more
            severe outcomes than personally experienced to encourage
            resource flows to perceived needs. Estimated outcomes tend
            to be more similar for people in Gaza, possibly because of
            greater homogeneity in socioeconomic experiences and
            exposure to violent conflict and broader interest in a
            survey on the effects of the May 2021 violence. The main
            results are robust to different ways of controlling for
            observable characteristics; neither alternative weights nor
            sample restrictions erase the systematic differences between
            the surveys.
display_title: A Tale of Two Surveys : Comparing
            the Outcomes of an In-Person and Web-Based Survey of Mental
            Health in the West Bank and Gaza
listing_relative_url: /research/2023/06/34099220/tale-two-surveys-comparing-outcomes-in-person-web-based-survey-mental-health-west-bank-gaza
topicv3: Mental Health m1326341 1430
publishtoextweb_dt: 2023-06-21T00:00:00Z
dois: 10.1596/1813-9450-10494
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
author_keys: 0: author_key: 1594966
1: author_key: 2096627
2: author_key: 1769409
model_extractions: Policy Research Working Paper 10494

# A Tale of Two Surveys

Comparing the Outcomes of an In-Person and Web-Based Survey of Mental Health in the West Bank and Gaza _Arden Finn Jessica Anderson Alia Aghajanian_ Poverty and Equity Global Practice June 2023
Policy Research Working Paper 10494

# **Abstract**

Internet surveys may never replace in-person surveys as a gold standard, but they remain important tools for rapid, remote, and low-cost data collection. The West Bank and Gaza Poverty and Equity team had a unique opportunity to compare a Facebook survey with an in-person survey covering conflict exposure and potentially associated socioeconomic and mental health outcomes over a similar time period. It is reasonable to expect that the estimates from internet surveys and in-person surveys would differ. In this case, the Facebook survey estimates more severe outcomes (e.g., higher exposure to conflict and worse mental health) than its in-person counterpart for most topics and populations. Multiple mechanisms may have contributed to this difference in estimates, including overrepresentation in the Facebook sample of respondents who were interested in the survey topics, reduced sensitivity bias in the context of a self-administered online questionnaire, and reporting more severe outcomes than personally experienced to encourage resource flows to perceived needs. Estimated outcomes tend to be more similar for people in Gaza, possibly because of greater homogeneity in socioeconomic experiences and exposure to violent conflict and broader interest in a survey on the effects of the May 2021 violence. The main results are robust to different ways of controlling for observable characteristics; neither alternative weights nor sample restrictions erase the systematic differences between the surveys.

This paper is a product of the Poverty and Equity Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at afinn1@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
A Tale of Two Surveys: Comparing the Outcomes of an In-Person and Web-Based Survey of Mental Health in the West Bank and Gaza<sup>1</sup> Arden Finn<sup>2</sup> Jessica Anderson Alia Aghajanian

**Keywords:** Survey Methodology, Internet Survey, Selection Bias, Sensitivity Bias, Conflict, Mental Health, West Bank and Gaza

JEL Classification: C81, C83

> 1 **Acknowledgments** : This report was prepared under the overall guidance of Johannes Hoogeveen and Alan Fuchs. Minh Cong Nguyen and the MENA Data Lab provided valuable support in setting up the Facebook survey and in calculating survey weights. We are grateful to Hamza Mighri (IMF) and Jawad al-Saleh (Palestinian Central Bureau of Statistics) for helpful inputs and suggestions in the early stages of planning the Facebook survey. Reviewer comments were received from Ganesh Seshan, Sharad Tandon and Hugo Nepo, and we are very grateful to these colleagues for flagging some analytical gaps, and for greatly improving the report.

> 2 Corresponding author: afinn1@worldbank.org
# Introduction

Internet surveys generally cannot match the coverage and quality of their more traditional in-person counterparts, but they remain important tools for rapid, remote, and low-cost data collection. This is particularly useful to track changes in response to a shock, as during the COVID-19 pandemic, or to provide an additional way to reach people when security or other circumstances preclude a face-to-face strategy, as in some fragile and violence-affected contexts. The self-administered online format may also encourage more honest responses to sensitive questions than are usually expected of an interviewer-administered survey. One case in which these attributes may be particularly valuable is in tracking exposure to violence and the various demographic, socio-economic, and mental health variables with which that exposure may be associated.

The West Bank and Gaza Poverty and Equity team had a unique opportunity to compare a Facebook survey with an in-person survey on these topics in 2022. Following the spike in violence in the West Bank and the attacks on Gaza in mid-2021, the World Bank co-led the production of a Rapid Damage Needs Assessment (RDNA). During this process it became clear that there was a large gap in the understanding of the mental health impacts on a population that is subject to continuous exposure to violence and conflict. To fill this gap, the West Bank and Gaza Poverty team together with local and international partners<sup>3</sup> designed and implemented the first nationally representative survey of mental health outcomes among Palestinians. The survey was conducted between March and May 2022. At the same time, the World Bank team ran a web-based survey on Facebook which matched, to the extent possible, the same questions contained in the in-person survey questionnaire. This Facebook survey built on the experience and expertise of the MENA Data Lab, which supported 15 online surveys advertised through Facebook in 2022/2023.

The main report on the in-person survey<sup>4</sup> outlines the relevant context of the West Bank and Gaza as well as theoretical frameworks linking mental health outcomes to socio-economic outcomes, experience of adversity, and exposure to violence and trauma. Taking this context and theoretical relationships as given, the analysis that follows explores differences between the in-person and Facebook surveys.

We can reasonably expect estimates from internet surveys and in-person surveys to differ, even if they aim to measure the same things. In the West Bank and Gaza example, the Facebook survey estimates more severe outcomes (i.e., higher exposure to conflict, worse mental health) then its in-person counterpart for most topics and populations. In this report we explore three possible mechanisms that may have contributed to this pattern of differences: (1) nonresponse bias, (2) sensitivity bias, and (3) misreporting to influence resource allocation.

We find that estimated outcomes tend to be more similar for people in Gaza, possibly because of greater homogeneity in socioeconomic experiences and exposure to violent conflict and broader interest in a survey on the effects of the May 2021 violence. We also assess the extent to which our results are robust to different ways of controlling for observable characteristics, and find that neither alternative weights nor sample restrictions erase the systematic differences between surveys.

> 3 These included the Palestinian Central Bureau of Statistics (PCBS), Zentrum Überleben, and the International Security and Development Center (ISDC).

> 4 World Bank Group et al (2022), available here. The PPCS data [World Bank (2023)] can also be accessed directly <u>here.</u>

2
# The surveys

The in-person Palestinians’ Psychological Conditions Survey (PPCS) aimed to better understand mental health outcomes in the West Bank and Gaza, as well as links between those outcomes and exposure to violence, socioeconomic conditions, and attachment to the labor market.<sup>5</sup> A detailed adult individual questionnaire embedded in a larger household survey included modules assessing personal exposure to conflict events (Gaza only), food security, signs of aggressive tendencies, life satisfaction, risk for mental health problems, symptoms of depression, exposure to traumatic events, symptoms of PTSD, COVID exposure, and prosocial behavior. The PPCS was fielded in person in the West Bank, East Jerusalem, and Gaza between March and May 2022. Out of a planned sample of 7,057 households, 6,140 participated in the survey. The data collection team attempted to interview 6,138 adults for the detailed individual interview, and 5,877 individuals formed the final sample used in analysis. A back-of-the-envelope calculation shows that each completed survey cost approximately $23.40, excluding staff and consultant time.

From March 19 to 22, 2022, concurrent with the first month of data collection for the PPCS, the Poverty team ran a pared down version of the adult individual questionnaire through a link that was advertised on Facebook. This shorter questionnaire covered basic sociodemographic questions, personal exposure to conflict events, food security, signs of aggressive tendencies, risk for mental health problems, symptoms of depression, COVID exposure, prosocial behavior, and questions about Facebook usage and whether any other household members had completed the survey (for weighting purposes). Adverts with the survey link were sent to a random sample of Facebook accounts stratified by governorate and gender to try to achieve a realized sample similar in those characteristics to the overall population. During the four days in which the Facebook adverts were live, 18,896 people clicked on the link. Of those, 2,279 people started the survey, and 1,227 respondents formed the final sample used for analysis. Each questionnaire in this sample cost approximately $1.30, excluding staff and consultant time.

**Table 1: The Facebook survey achieved a smaller sample size but was faster and less**

**expensive than the PPCS**

||**PPCS**|**Facebook survey**|
|---|---|---|
|_Data collectionperiod_|7 Mar-9 May2022|19-22 Mar 2022|
|_Final sample size_|5,877|1,227|
|_Approximate costper survey infinal sample_|$23.40|$1.30|

The Facebook survey was structured similarly to the PPCS to try to maximize comparability. The core content modules included in the Facebook survey largely followed the order used in the PPCS and were generally based on the same English questions and response scales. The different modalities used by the two surveys imply potential differences in interpretation, and slight differences in the wording and tone of the Arabic text could also have influenced interpretation. We do not believe these differences would change our key findings but include further detail on the questionnaires in Appendix A. To the extent possible, the Facebook survey weights were calibrated using similar marginal population totals to those used to calibrate the PPCS weights.

> 5 For further information, see World Bank Group et al (2022).

3
The Facebook survey consistently estimated more severe outcomes than the in-person survey The national-level Facebook estimates range from about 14 percent to about 250 percent more severe than their PPCS counterparts (see Table 2). For six of the eight core indicators, the difference in estimates between the two surveys is significant at the 5% level. The direction of difference holds in almost all cases if estimates are disaggregated by region, gender, age category (18 to 34 / 35 and older) or reported exposure to at least one conflict event. Estimates do not appear to be systematically more similar for any of the subgroups created by these disaggregations, except for respondents from Gaza. This finding is discussed in more detail in a later section. Nor does the direction of differences between subgroups in the Facebook survey necessarily match the direction of differences between the same subgroups in the PPCS.

**Table 2: The Facebook survey estimated more severe outcomes than the PPCS across eight core**

**indicators**

||**Indicator**|**Indicator**<br>**range **|<br>**PPCS**<br>**estimate**|**Facebook**<br>**estimate**|<br><sup>**Difference**</sup>|**N**<br>**(combined)**|
|---|---|---|---|---|---|---|
|_1_|_Share of people who report personal_<br>_exposure to at least one conflict event_<br>_(Gaza only)_|0-100%|35.0<br>(3.2)|49.7<br>(8.8)|14.7<br>(9.4)|2937|
|_2_|<sup>_Approximate prevalence of moderate or_</sup>|0-100%|29.0|67.1|38.1|5703|
||_severe food insecurity_||(2.1)|(7.0)|(7.4)||
|_3_|_Average Reduced Coping Strategies Index_|0-56|5.7|20.0|14.3|6408|
||||(0.4)|(2.7)|(2.7)||
|_4_|_Average aggressive tendencies index_|1-5|1.9|2.7|0.8|6791|
||||(0.0)|(0.1)|(0.1)||
|_5_|<sup>_Average risk for mental health problems_</sup><br>_index_|1-4|2.0<br>(0.0)|2.4<br>(0.1)|0.4<br>(0.1)|6779|
|_6_|<sup>_Prevalence of symptoms consistent with_</sup>|0-100%|57.7|87.4|29.8|6837|
||_depression_||(2.2)|(2.6)|(3.4)||
|_7_|_Averae deression smtom severit level_|0-100|54.8|72.8|18.0|6837|
||_g p yp y_||(1.0)|(2.4)|(2.6)||
|_8_|<sup>_Share of people who report at least one_</sup>|0-100%|72.5|82.6|10.1|6808|
||_type of exposure to COVID-19_||(2.2)|(5.8)|(6.2)||

Notes: For all indicators, higher values indicate more severe outcomes. Indicator 2 is based on the Food Insecurity Experience Scale (FIES); Indicator 4 is based on the 12-item short form of the Buss-Perry Aggression Questionnaire (BPAQ-SF); Indicator 5 is based on 12-Item General Health Questionnaire (GHQ-12); Indicators 6 and 7 are based on WHO-5 Well-Being Index (WHO5). See World Bank Group et al (2022) for further detail on indicator choice and construction. All indicators except conflict and COVID exposure include only complete responses to the module; conflict and COVID exposure indicators include partial responses to the module. Clustered standard errors are reported in parentheses.

This systematic difference in overall results could stem from multiple sources. Part of it is likely driven by who makes up the Facebook survey sample. A small fraction of the people who clicked on the Facebook survey link completed the questionnaire, and this sample almost certainly overrepresents people who are relatively more interested in the psychological conditions of Palestinians and the effects of the May 2021 4
violence. Systematic variation in response patterns could also have contributed to the divergence. People experiencing severe outcomes that are seen as socially undesirable—aggressive tendencies or indicators of risk for mental health problems, for example—may have been more likely to have reported their experiences honestly in a self-administered online questionnaire than in an in-person interview. A different set of respondents, who were not themselves experiencing severe outcomes but perhaps had friends or relatives in difficult circumstances, may purposely have reported more severe outcomes than they experienced personally with the goal of influencing resource allocation toward the needs they observed. While the PPCS and Facebook survey were not designed to isolate the effects of these different mechanisms, we explore each of them further in the sections that follow.

Were people who experienced more severe outcomes more likely to complete the Facebook survey?

The PPCS team secured high response and completion rates for the detailed adult individual interview, but the Facebook survey reflected the nonresponse and dropout challenges that are common to online questionnaires. Almost 96 percent of the adults selected for the PPCS individual interview agreed to participate, and virtually all of those adults completed the questionnaire. In contrast, about 12 percent of the people who clicked on the Facebook-advertised link (themselves a fraction of the Facebook users for whom the advertisement appeared) agreed to start the survey, and of those respondents only 54 percent remained to comprise the final sample.

The Facebook and PPCS samples differ substantially in some observed characteristics (further discussed below in the section on weighting strategies), and it is likely that they also differ in important unobserved characteristics given the high nonresponse rates of the Facebook survey. One plausible unobserved selection mechanism is interest in the survey topics. According to the introduction to the Facebook survey that preceded agreement to participate, the questionnaire was “about the psychological conditions of Palestinians living in the West Bank and Gaza” and the research team was “trying to understand how people have been affected by the conflict that took place last May.” Other topics mentioned include background information, employment status, food security, and “exposure and reactions to recent events.” The Facebook survey almost certainly achieved higher response rates among people for whom these topics—particularly the psychological conditions of Palestinians and effects of the May 2021 conflict— were interesting and relevant. This likely included people who had experienced the effects of the May 2021 conflict at close range (including through the experiences of friends and relatives), potentially inducing selection bias in measures of conflict exposure and associated variables. The associations between experiences of conflict exposure, mental health, and food insecurity are nuanced: mental health responses to adverse events vary from person to person, and the specific conflict events investigated in the Facebook survey represent a small subset of the overall conditions of direct and structural violence in the West Bank and Gaza. However, given the observed differences in estimates between the two surveys, we think it possible that the people who chose to respond to the Facebook survey included a higher proportion (relative to the population) of those who experienced greater exposure to the May 2021 escalation as well as more severe outcomes in the other survey topics.

While the introductory letter for the PPCS included a similar description of the core topic of the survey, nonresponse bias related to topic interest should be much smaller in this case because almost all adults 5
<!-- Start of picture text --> 100%<br>BO%<br>2<br>a<br>3<br>@<br>x<br>20%<br>o%<br>einSS555323 a838 €F iF §$35 gg §iu<br>3 ~ | = ba a B 5 S<br>*<br>5 of Zu 82 ps FA 3 9 s<br>2 = rg a 3 3<br>§ : 8 if<br>: 2<br>3<br>gz<br><!-- End of picture text -->
It is difficult to say with certainty why respondents to the Facebook survey dropped out at these points, but the partial data on reported conflict exposure suggest that dropout varied with experience of conflict events (see Table 3). We focus on respondents from Gaza for all analysis of reported conflict exposure because only Gazan respondents were asked these questions in the PPCS, but the pattern of results reported in Table 3 is not different if we use the full national sample. Among respondents from Gaza, respondents who reported no exposure to conflict events were overrepresented among those who dropped out of the survey after partially completing it. This pattern is even more pronounced among respondents who dropped out of the survey immediately after the conflict exposure module. Overall, respondents from Gaza who reported experiencing at least one conflict event were about 6.5 percent more likely to complete the survey than those who did not report experiencing any conflict events, which is consistent with hypothesis H1a.

# **Table 3: Respondents who reported no exposure to conflict events were overrepresented among those who dropped out of the Facebook survey**

||**Respondents who reported no**<br>**exposure to conflict events**|
|---|---|
|_Share of all respondents_|37.6%|
|_Share of respondents who dropped out at any time before_<br>_completing the survey_|40.5%|
|_Share of respondents who dropped out at the end of the conflict_<br>_exposure module_|46.0%|

Note: Restricted to respondents from Gaza who answered at least one conflict exposure question.

Were people more willing to provide answers to sensitive questions in the Facebook survey than in the PPCS?

Systematic variations in response patterns may also have contributed to the differences between the PPCS and Facebook survey estimates. One potential variation of this type is sensitivity bias, which could stem from multiple sources.<sup>6</sup> For example, respondents might underreport aggressive tendencies because these tendencies conflict with their self-image, because they fear the consequences if their reporting is disclosed, or because they do not wish to be associated with a socially undesirable trait. They may underreport signs of risk for mental health problems for fear of disclosure or perception of social undesirability, or because mental health problems are considered taboo topics that should not be discussed with others. For the core topics shared between the PPCS and the Facebook survey, we would generally expect response options indicating more severe outcomes to be more sensitive.

Existing evidence suggests that self-administered internet surveys may reflect lower sensitivity bias relative to in-person and telephone surveys, which could be consistent with the pattern of more severe outcomes estimated from the Facebook survey relative to the PPCS. For example, Kreuter et al (2008) found that online self-administration increased accurate reporting of sensitive information relative to computer-assisted telephone interviewing, and Tandon and Vishwanath (2022) presented evidence that anonymous internet-based surveys elicited sensitive information more accurately than a concurrent mobile phone survey. Under the social reference theory developed by Blair et al (2020), self-

> 6 Primarily based on the sources of sensitivity described in Blair et al (2020) and Isaqzadeh et al (2020).

7
administration techniques can reduce sensitivity bias by changing the social context in which sensitive questions are asked so that the identity of the respondent is obscured from interviewers, bystanders, and other potential social referents.

While most of the core topics shared between the PPCS and the Facebook survey could have been considered sensitive, we expect that in this context the questions on conflict exposure and mental health may have been more sensitive than the questions on food insecurity and COVID-19 exposure. Respondents may have perceived the questions on personal exposure to conflict to be sensitive because they brought up memories of traumatic experiences. In the mental health modules, respondents may have perceived questions as sensitive because of the negative attitudes and stigma around mental illness that persist in many parts of the region.<sup>7</sup> Questions about food insecurity may be perceived as sensitive because of their association with poverty, but we expect this sensitivity to be reduced in the specific context of the PPCS because most respondents had participated in the last wave of the Socio-Economic and Food Security Survey and might therefore be accustomed to answering questions about food insecurity. COVID-19 restrictions are not as severe as they were in earlier stages of the pandemic, and surveys about people's experience of COVID have proliferated since early 2020. We might therefore expect the difference in sensitivity bias between the PPCS and the Facebook survey to be more marked in the indicators related to conflict exposure and mental health than in the indicators related to food insecurity and COVID-19 exposure.

# _Hypotheses_

H2a: People tend to underreport conflict exposure, aggressive tendencies, indicators of risk for mental health problems, and symptoms consistent with depression because these are perceived as dangerous or undesirable traits and experiences.

H2b: People are more likely to underreport dangerous or undesirable traits and experiences in an inperson survey than in a self-administered web survey because the interviewer and bystanders are important social referents for respondents.

# _Discussion_

We explore this pair of hypotheses by comparing shares of sensitive responses reported in the Facebook survey and in the PPCS—following the approach of Tandon and Vishwanath (2022). We define sensitive responses for this purpose as the response options that indicate personal exposure to conflict events, stronger aggressive tendencies, higher risk for mental health problems, and more severe depression symptoms. We would expect to see higher shares of these response options chosen in the Facebook survey than in the PPCS.

If sensitivity bias were the strongest mechanism contributing to the differences we observe between the Facebook and PPCS estimates, we might also expect to see larger differences between the two surveys in the shares of sensitive responses to the conflict exposure and mental health questions than in the shares of responses indicating more severe outcomes in the modules we expect to be less sensitive (food insecurity and COVID-19 exposure).

> 7 See for example Alyafei et al (2021) and Elyamani et al (2021) for references to negative attitudes and stigma in regional context, or World Health Organization: Regional Office for the Eastern Mediterranean (n.d.) for reference to stigma in the Palestinian context.

8
However, we do not find evidence of larger between-survey differences in the topics we expected to be more sensitive (see Table 4). We find substantial differences in both sets of modules, and the comparison suggests if anything larger differences in the topics we expected to be less sensitive. This is driven by the two food insecurity modules, and in fact COVID-19 exposure is the module for which item-level results are most mixed. It is possible that our expectations about which topics would be more and less sensitive in this context were inaccurate, and food insecurity questions belonged in the more sensitive class. It could also be that accessing the online questionnaire from Facebook limits its perceived anonymity and reduces the advantage of the online mode for honest reporting. Finally, it is entirely possible that other mechanisms played a larger role in shaping the pattern of differences between the two surveys, or that the mechanisms had varying effects across modules.

**Table 4: Reduced sensitivity bias in the Facebook survey may have contributed to the differences between surveys, but we cannot provide clear evidence of its contribution**

||**Indicator**|**Shar**<br>**response**<br>**more sev**<br>_PPCS_|**e choosing**<br>**s that indicate**<br>**ere outcomes**<br>_(%)_<br>_Facebook_|**Difference**<br>_(percentage_<br>_point)_|
|---|---|---|---|---|
||_A1_Average share of respondents choosing responses<br>that indicate personal exposure to conflict events|7.47|24.17|16.70|
|_itive_|_A2_Average share of respondents choosing responses<br>that indicate stronger aggressive tendencies|20.49|33.19|12.70|
|_e sens_|_A3_Average share of respondents choosing responses<br>that indicate higher risk for mental health problems|15.89|34.00|18.11|
|_Mor_|_A4_Average share of respondents choosing responses<br>that indicate higher severity of depression|61.20|69.26|8.06|
||symptoms||||
||_Average across indicators_|_26.26_|_40.15_|_13.89_|
|_ve_|_B1_Average share of respondents choosing responses<br>that indicate food insecurity (based on FIES)|34.28|54.31|20.03|
|_sensiti_|_B2_Average share of respondents choosing responses<br>that indicate food insecurity (based on RCSI)|30.02|55.46|25.44|
|_Less_|_B3_Average share of respondents choosing responses<br>that indicate exposure to COVID-19|38.44|46.12|7.68|
||_Average across indicators_|_34.23_|_50.79_|_16.56_|

Notes: The indicators reported here are related but not equivalent to the index/score and prevalence indicators elsewhere in the report. Responses indicating more severe outcomes are constructed as responses of “Yes” in a binary response scale (A1, B3); as all response options on the more-severe-than-neutral side of response scales with a clear neutral/tipping point (A2-A4); or as responses indicating any experience of food insecurity (B1-B2). The unweighted share of respondents choosing responses that indicate more severe outcomes is estimated for each question, then averaged across all non-contingent questions in each topic module. Item-level results are reported with clustered standard errors in Appendix B. In all modules except COVID-19 exposure, all or most item-level differences are positive and significant at the 5% level at least.

9
Did some people purposely misreport outcomes in the hopes of influencing resource allocation?

While some respondents who experienced severe (and socially undesirable) outcomes may have reported them more honestly in the relative anonymity of an online survey, a different set of respondents may purposely have reported more severe outcomes than they experienced to try to influence resource allocation. This type of misreporting has not been widely studied, but Kaplan et al (2019) offer evidence that some vulnerable Internally Displaced Persons (IDP) households in South Sudan may underreport consumption in surveys. They link this behavior to a desire to influence the allocation of humanitarian assistance, and find that light-touch honesty primes reduce the gap in reported consumption between IDP and non-IDP households for respondents more likely to be underreporting and in measures more easily manipulated.<sup>8</sup> The framing and contexts of the PPCS and Facebook surveys suggest that strategic misreporting to influence resource allocation may have been more prevalent in the Facebook survey than in the PPCS. The role of the World Bank was prominent in the framing of the Facebook survey, with World Bank Surveys sponsoring the Facebook advert and the Bank logo branding the online questionnaire. In the previous year, the Bank had co-led a Rapid Damage Needs Assessment that informed international assistance strategies. Introductory text to the Facebook survey noted that the Bank wanted to understand how people had been affected by the May 2021 violence, but did not mention the importance of accuracy. By contrast, the PPCS was implemented by enumerators from the national statistical office (PCBS). The detailed adult individual interview inquiring about conflict exposure and mental health outcomes was embedded in a larger household survey, and 91 percent of PPCS adult respondents had previously responded to the Socio-Economic and Food Security Survey 2018 individual module (itself a continuation of a panel that started in 2013). At no point in this panel were respondents offered incentives to participate. The introductory letter sent to potential participants in the PPCS also emphasized the importance of providing accurate information.

# _Hypotheses_

H3a: Some people report more severe outcomes than they experience personally because they hope to influence resource allocation toward addressing these problems.

H3b: This effect was stronger in the Facebook survey than in the PPCS because in the Facebook survey the role of the World Bank was more prominent and the incentives to accurately report personal experience were weaker than in the PPCS.

# _Discussion_

Respondents who misreport their experiences in this way seem likely to be people who are not themselves experiencing severe outcomes but who have seen friends, relatives, or other acquaintances experiencing severe outcomes. The effect is perhaps less likely to have contributed to differences in the estimates of aggressive tendencies, as perceived aggression may discourage assistance. Beyond that, it is difficult to separate out this mechanism from other sources of difference in our data.

> 8 Purposeful misreporting and strategic answering behaviors are also explored in De Juan & Koos (2021) and Lund et al (2011).

10
Estimates tend to be more similar between the two surveys for people in Gaza Within the overall pattern of differences, estimates disaggregated by region are more similar between the PPCS and the Facebook survey in Gaza than in the West Bank for six of the seven core indicators calculated for both regions (conflict exposure is excluded here because it is calculated only for Gaza). For these six indicators, the greater degree of similarity is substantial: the relative difference between the two surveys tends to be about half as much in Gaza as it is in the West Bank. (For the one indicator in which estimates are more similar in the West Bank, the differences in estimates are close between the two regions.)

**Figure 2: The relative difference between Facebook and PPCS estimates is smaller in Gaza than in the West Bank**

This may be in part because there is more homogeneity in Gaza around geography, socioeconomic characteristics, and exposure to conflict than there is in the West Bank. For example, the West Bank is more than 15 times the size of Gaza and more varied in the distribution of its population over type of location (urban, rural, or camp). Per capita welfare is also more variable in the West Bank than in Gaza: the sample standard deviation of the most recent per capita welfare aggregate is about 1.5 times larger in the West Bank than in Gaza. Estimated poverty rates vary more within the West Bank, with a difference of about four percentage points between the poorest and wealthiest governorates in Gaza compared to about 10.5 percentage points between the poorest and wealthiest governorates in the West Bank (this difference in variability increases at lower levels of disaggregation).<sup>9</sup> Experience of violent conflict also tends to vary more across the West Bank than it does in Gaza, as suggested by the pattern of events recorded by the Armed Conflict Location and Event Data Project (ACLED) during the May 2021 violence. Over a period of 15 days, ACLED recorded fewer than 15 total events involving violence in four West Bank governorates, while the highest governorate-level event counts in the West Bank in that same period

> 9 For further detail on this, see Atamanov & Palaniswamy (2019).

11
reached 90 or more. Total event counts by governorate in Gaza fall within a narrower range, from 30 at the lowest to almost 100 at the highest.<sup>10</sup> Not only does the distribution of outcomes tend to be more condensed in Gaza than in the West Bank, but those outcomes are on average more severe. If the Facebook survey tends to pull estimates toward the more severe end of the scale relative to the PPCS, the measurable differences produced by this process are likely to be smaller if the PPCS estimates are already at that end of the scale.

People in Gaza are also plausibly more likely to be interested in responding to a survey on the effects of the May 2021 violence on Palestinians, suggesting that the sample of respondents who selected into the Facebook survey is likely to be more representative of the underlying population in Gaza than it is in the West Bank. May 2021 saw escalations of violence in both Gaza and the West Bank, but Gaza experienced more severe violence and more extensive damage.<sup>11</sup> This greater exposure may have resulted in broader motivation to complete a World Bank Facebook survey seeking to understand the effects of the escalation. In fact, out of the sample of respondents who began the Facebook questionnaire, people in Gaza were over 25 percent more likely to complete the survey than were people in the West Bank.

Finally, all respondents to the Facebook survey received the conflict exposure questions, while only respondents in Gaza received these questions in the PPCS. Since the conflict exposure module preceded the mental health modules, some of the greater similarity in Gaza for these modules may be due to the regional differences in questionnaire structure.

How robust are the main results to different ways of controlling for observable characteristics?

# Achieved samples

As is often the case with online surveys, the achieved sample for the Facebook survey differed from the population on several observable characteristics (see Table 5). Respondents to the Facebook survey were more likely than the general population to live in the West Bank, more likely to be male, more likely to report working in the past week, and much more likely to have completed secondary or higher education. The distribution of observable characteristics in the PPCS sample was more closely (although not perfectly) aligned with that of the population.

**Table 5: The achieved sample for the Facebook survey is farther than the PPCS sample from the population distribution**

|**Indicator**|**Facebook sample**<br>**(unweighted)**|**PPCS sample**<br>**(unweighted)**|**Population**|
|---|---|---|---|
|_Share who live in the West Bank_|69.4|56.4|62.0|
||(1.3)|(2.6)||
|_Share who are female_|24.5|51.8|49.4|
||(1.2)|(0.5)||

> 10 Based on publicly available <u>ACLED data for the West Bank and Gaza from May 6 to 21, 2021, accessed around</u> December 2022 and filtered to the following event and sub event types: Battles, Explosions/Remote violence, Riots, Violence against civilians, and Excessive force against protesters. See also Raleigh et al (2010). 11 See for example Mehvar et al (2021) and World Bank Group et al (2021).

12
|_Share who are 35 and older_|59.3|65.8|48.8|
|---|---|---|---|
||(1.4)|(0.7)||
|_Share who have less than secondary_|4.1|54.4|55.1|
|_education_|(0.6)|(1.0)||
|_Share who report working in past week_|59.7|37.0|31.9|
||(1.5)|(0.7)||
|_Share who are refugees_|46.5|42.1|42.2|
||(1.4)|(2.0)||

Notes: All survey sample shares are of individuals aged 18 and older. Clustered standard errors are reported in parentheses for survey estimates. Population shares by region, gender, and age are derived from population projections provided by PCBS in early 2022 and refer to individuals aged 18 and older. Population shares by educational attainment are based on population projections published by PCBS for 2021 and refer to individuals aged 15 and older. Population shares by labor force status originate in the 2021 Palestinian Labor Force Survey and refer to individuals aged 15 and older; the share reported here is of employed individuals. Population shares by refugee status come from 2017 estimates published by PCBS and refer to all individuals.

# Precision-comparability tradeoffs in calibrating the Facebook weights to population proportions

The initial weighting strategy for the Facebook survey imposed a substantial set of calibration constraints to try to maximize comparability with the PPCS and correct for a skewed achieved sample. PPCS weights were calibrated to population proportions using two sets of constraints based on population projections based on the most recent census: (1) the number of households by governorate and urban/rural/camps, and (2) the number of adults by region, five-year age groups, and gender. The weighting approach for the Facebook survey followed a slightly simplified version of this calibration structure using population proportions by governorate, ten-year age groups, and gender (marginal totals only). It also added educational attainment as a constraint to correct for the overrepresentation in the raw sample of Palestinians who had completed secondary or higher education.

This may have been an overly ambitious calibration structure to impose on the Facebook survey sample size, as it resulted in very small cell sizes and widely distributed weights. For example, the sample to which weights were assigned contained only one man aged 18 to 24 years living in Jenin with primary education. The cells created by the initial calibration constraints range in size from 1 to 38 respondents, with almost 80 percent of the cells having 5 or fewer respondents. The standard deviation of the resulting weights is almost 5.5 times that of the PPCS weights (8,439.25 relative to 1,534.971).

Given these drawbacks of the initial calibration approach, a reasonable alternative might calibrate the Facebook survey weights only on region (West Bank/Gaza) and gender—sacrificing some comparability for decreased variability and increased precision. Weights calibrated using this more minimalist set of constraints have a standard deviation about one-third as large as the standard deviation of weights calibrated using the more ambitious constraints, and a median weight a little over four times the median weight of the initial set.

For most of the core indicators, the alternative weights tend to reduce the difference between the Facebook estimates and the PPCS estimates without changing the qualitative direction of difference. The exception to this trend is in the share of people (in Gaza) who report personal exposure to at least one 13
conflict event, where estimates using the alternative weights are higher and therefore farther away from the PPCS estimates.<sup>12</sup>

## An unweighted approach

As an alternative way to try to control the distribution of observable characteristics, we sequentially restrict both survey samples to three groups that are overrepresented in the full Facebook sample: people with secondary or higher education, men, and people who report working in the past week. We then reestimate the main indicators from Table 2 using each pair of restricted samples, now without weights. We compare these estimates to those generated using the full (unweighted) samples for each survey. None of the three sample restrictions significantly alter the between-survey differences estimated for the main indicators using the full (unweighted) samples. Estimates of the between-survey differences in the main indicators for all four specifications are included in Appendix C.

# Conclusion

Given the opportunity to compare a Facebook survey to an in-person survey covering conflict exposure and potentially associated socio-economic and mental health outcomes in the West Bank and Gaza over a similar time period, we found that the Facebook survey consistently estimated more severe outcomes for a variety of indicators. Multiple mechanisms may have contributed to this difference in estimates, including overrepresentation in the Facebook sample of respondents who were interested in the survey topics, reduced sensitivity bias in the context of a self-administered online questionnaire, and reporting more severe outcomes than personally experienced to encourage resource flows to perceived needs.

Estimates were more similar between the surveys in Gaza, where the population is more homogeneous in relevant characteristics and more likely to be interested in responding to a survey on the effects of the May 2021 violence. The overall pattern of more severe outcomes estimated from the Facebook survey is robust to different ways of controlling the distribution of observable characteristics.

### **To what extent might these results generalize to other internet surveys?**

Most internet surveys are likely to produce results that differ meaningfully from those estimated by an inperson survey. Most are also likely to find that their achieved samples are biased on important sociodemographic characteristics relative to the general population, but these observable differences may or may not explain the differences in estimated outcomes.

The three mechanisms that we explore to explain the differences we estimate are good candidates to affect other internet surveys, but it should not be assumed that they will always move internet estimates in the same direction or to the same extent. For example, many internet samples will overrepresent people who are interested in the survey topics. In the case we discuss, we suggest that selection into the survey based on topic interest was correlated with more severe outcomes – that is, people were more

> 12 The change in the Facebook estimates that results from switching to the more minimalist set of weights is closely associated with differences between outcomes for more and less educated respondents, as the strongest correction that the initial calibration constraints make is to the distribution of educational attainment.

14
likely to complete our Facebook survey if they had personally experienced challenges related to the survey topics. This may often be the case for internet surveys that explicitly focus on problems or challenging circumstances experienced by the respondents. However, not all internet surveys estimate more severe outcomes than their more traditional counterparts. For example, a recent working paper by Soundararajan et al (2023) finds that across six countries Random Domain Intercept Technology (RDIT) surveys tend to overestimate employment-to-population ratios.

In addition, the extent of potential selection and misreporting bias is likely to be affected by survey framing as well as topic and context. Some features of the Facebook survey framing in our example may have exacerbated selection and misreporting bias, and careful design of future internet surveys could reduce that contribution.

# **Taking advantage of opportunities for comparison to establish a baseline**

Internet surveys may be useful for rapid assessment, with the understanding that they will present an incomplete picture relative to the overall population. The MENA Data Lab, for example, has supported a handful of surveys on food insecurity and price expectations across multiple countries in North Africa amid recent concerns over inflation. In contexts in which volatility is expected, it could be valuable to take advantage of opportunities to pair a more traditional survey with a simultaneous online survey covering the same content. This is relatively simple and inexpensive to do, and it establishes a baseline for future online surveys in the event of a shock or otherwise rapidly changing circumstances.

# **Maximizing the potential of internet surveys to triangulate responses to sensitive questions**

Internet surveys may be able to add value to their more traditional counterparts by offering a different perspective on sensitive questions based on more honest reporting, but that value is reduced if we cannot figure out how much of the difference between traditional and internet estimates is due to more honest reporting and how much is due to other factors.

To help isolate differences due to more honest reporting and minimize other sources of difference, future comparison efforts might consider:

- Assessing at the design stage which other mechanisms might operate alongside sensitivity bias to produce differences between the planned surveys, what can be predicted about their relative size and direction, and the extent to which they could be minimized in the survey design

- Including a sufficient set of non-sensitive (or at least substantially less sensitive) questions to be able to compare between-survey differences in sensitive and non-sensitive questions

- Framing introductory materials in general terms that avoid presenting potentially sensitive topics as the focus of the survey

# **Future questions to explore**

Further research into the potential value of internet surveys for sensitive topics could compare the results of an online questionnaire advertised on Facebook or other social media platform with one collected using RDIT to assess whether accessing the survey via a social media platform reduces respondent perceptions of anonymity. The online survey modality also offers low-cost opportunities to investigate interactions between questionnaire design and sensitive response behavior, for example by randomizing the order in which modules are presented to test for potential framing effects.

15
It would additionally be useful to explore the extent to which selection/nonresponse bias might be mitigated in an online survey by varying the introductory language to the questionnaire and increasing the salience of other features that might affect survey participation. To try to gauge the potential effects of misreporting, future survey experiments might include a scenario question designed to prime the value of honesty or alter the questionnaire branding and introduction to vary the extent to which the survey is linked to national statistical processes versus international assistance planning.

Internet surveys may never replace in-person surveys as a gold-standard tool for data collection, but investigating questions like these could help us learn how to maximize the unique advantages they offer.

16
# Appendix A: Questionnaires

The first table below summarizes the structure of the questionnaires. Subsequent tables reproduce the English text used in the Facebook questionnaire for each substantive module, as well as notes on the comparability of the two questionnaires for that module. Both surveys were delivered entirely or almost entirely in Arabic, and while the Arabic text of the questionnaires is usually similar, it is rarely exactly the same. Substantive or systematic differences are noted in the module summaries that follow.

|**PPCS**|**Facebook survey**|
|---|---|
|Household questionnaire covering sociodemographic<br>details, housing, assistance and coping strategies,<br>consumption, household income, and mobility and<br>access to services||
||Q1. Agreement toparticipate|
|S1. Identification information: name, contact<br>information, marital status, sex, age|Q2-16. Sociodemographic questions: age,<br>region/governorate, gender, education, refugee<br>status, marital status, household size, relationship to<br>head of household,rooms in dwelling,work status|
|S2. Evaluation of services||
|S3. Conflict exposure – Gaza only|Q17-23. Conflict exposure(all respondents)|
|S4. Food Insecurity Experience Scale module|Q24-25. One of two food security modules (randomly<br>assigned): Food Insecurity Experience Scale or<br>Reduced CopingStrategies Index|
|S5. Aggression|Q26. Aggression|
|S6. Satisfaction||
|S7. General health questionnaire<br>WHO-5 depression module<br>Additional questions about mental health,<br>cognition, and behavior|Q27-28. General health questionnaire<br>Q29. WHO-5 depression module<br>[Information about Gaza Community Mental Health<br>Program]|
|S8. Traumatic events and PTSD||
|S9. COVID exposure||
|S10. Prosocial behavior|Q30. Prosocial behavior|
||Q31. COVID exposure|
||Q32-35. Weighting questions|

## **<mark>Personal exposure to conflict events</mark>**

|_English text of Facebookquestionnaire_||_Notes on comparability_|
|---|---|---|
|17<br>Have you been injured due to Israeli airstrikes or clashes<br>with Israeli securityforces in May2021?|Yes / No|Questions 18-23 were differently<br>ordered in the PPCS.|
|Has a household member of yours died due to Israeli|Yes / No||
|airstrikes or because of clashes with Israeli security forces||In the Arabic text of the PPCS, the|
|in May2021?||detailed phrasing about Israeli|
|Was any other household member injured in Israeli|Yes / No|airstrikes or clashes with Israeli|
|airstrikes or clashes with Israeli security forces in May||security forces in May 2021 is|
|2021?||replaced byaphrase approximately|

17
|Did an extended family member or close friend die due to<br>Israeli airstrikes or clashes with Israeli security forces in<br>May2021?|Yes / No|translating to “the May 2021<br>aggression”. In addition, the two<br>surveys used different Arabic words|
|---|---|---|
|Was an extended family member or close friend injured<br>in Israeli airstrikes or clashes with Israeli security forces in<br>May2021?|Yes / No|for “died”; the word used in the<br>PPCS is related to martyrdom.|
|18<br>Was your workplace demolished or damaged due to<br>Israeli airstrikes in May 2021?|Yes / No /<br>Prefer not<br>to answer|Questions 18 and 21 had to be<br>marked as required questions in the<br>Facebook survey for the skip|
|19<br>Is the company/entity still functioning?|Yes / No /<br>Don’t know|patterns to function correctly. To<br>avoid forcing respondents out of|
|20<br>Areyou still workingin the same company/entity|Yes/No|the survey at these points, a “Prefer|
|21<br>Was your home destroyed or damaged due to Israeli<br>airstrikes in May 2021?|Yes / No /<br>Prefer not<br>to answer|not to answer” response option<br>was added to each question. These<br>response options were not|
|22<br>Did you have to move because the house was not<br>habitable?|Yes / No|available in the PPCS.|
|23<br>How close was the nearest explosion to your then-home<br>in May2021?|[In<br>Kilometers]||

# **<mark>Food Insecurity Experience Scale (FIES) module</mark>**

# _<mark>English text of Facebook questionnaire</mark>_

- 24 We would now like to ask you about food and meals over the last 30 days. During the past 30 days, how many times has your household experienced the following? Response options: None, Once or twice, 3 to 10 times, More than 10 times, Don’t know/no answer

- a) You worried that the household would not have enough food

- b) You or any household members were not able to eat the kinds of foods you prefer because of a lack of resources

- c) You of any household member ate a limited variety of food because of a lack of resources

- d) You or any household member ate a smaller meal than you felt you needed because there was not enough food

- e) You or any household member ate fewer meals a day because there was not enough food

- f) There was no food at all in your household because there were no resources to get more

- g) You or any household member go to sleep at night hungry because there was not enough food

# _<mark>Notes on comparability</mark>_

The PPCS included two versions of the FIES module, one in the household questionnaire and one in the 18+ individual questionnaire. We use the module from the household questionnaire for this analysis because the frame of reference and response options match those used in the Facebook survey. The FIES module in the PPCS 18+ individual questionnaire refers to the individual (rather than household) experience over the past 12 months, with binary yes/no response options.

The FIES module in the PPCS household questionnaire included an extra item between statements (c) and (d).

Respondents to the Facebook survey were randomly assigned to receive either the FIES module or the RCSI module (described below).

- h) You or any household member went for a whole day and night without eating because there was not enough food

# **<mark>Reduced Coping Strategies Index (RCSI) module</mark>**

_<mark>English text of Facebook questionnaire</mark>_

- 25 During the last 7 days, when the household did now have enough food or money to buy the food, how many days did you... Response options: 0 day, 1, 2, 3, 4, 5, 6, 7 days

_<mark>Notes on comparability</mark>_ The RCSI module in the PPCS is located in the household questionnaire rather than in the 18+ individual questionnaire.

18
- a. Reduced the number of meals for all household members per day

- b. Reduced the quantity of meals eaten by adults in favor of children

- <mark>c. Purchased low quality markets “Leftover”</mark>

- d. Reduced portion of food for adults in favor of children's

Respondents to the Facebook survey were randomly assigned to receive either the FIES module (described above) or the RCSI module.

In the PPCS, the Arabic text for item (d) translates approximately to “reduced amount of food in one meal for all family members”.

- <mark>e. borrow food or rely on help from family and friends</mark>

- f. Refrain from consuming expensive and resort to alternatives (buying cheaper kind of food)

# **<mark>Aggression</mark>**

# _<mark>English text of Facebook questionnaire</mark>_

- 26 We would now like to ask you about how like or unlike you some actions are. Using the 5-point scale, indicate how uncharacteristic or characteristic each of the following statements are in describing you? Response options: 1. Not like me at all 2. Not much like me 3. Neutral 4. somewhat like me 5. very like me

- <mark>a) Given enough provocation, I may hit another person. b) I often find myself disagreeing with people.</mark>

- <mark>c) At times I feel I have gotten a raw deal out of life.</mark>

- d) There are people who have pushed me so far that we have come to blows.

- e) I can't help getting in arguments when people disagree with me.

- <mark>f) Sometimes I fly off the handle for no good reason.</mark> <u><mark>g)</mark></u> <mark>Other people always seem to get the breaks.</mark>

# _<mark>Notes on comparability</mark>_

In the Arabic text of the PPCS, the introductory question ends with “as a result of the May 2021 aggression”.

The Arabic text of the PPCS response scale uses more complicated phrasing than does the Arabic text of the Facebook scale. For example, response option 2 translates roughly to “this behavior usually does not apply to me”.

There are more substantive differences than usual between the Arabic texts of the two surveys in this module, largely in translating the more idiomatic English phrases.

- <mark>h) I have threatened people I know.</mark>

- <mark>i) My friends say I am somewhat argumentative.</mark>

- <u><mark>j)</mark></u> <mark>I have trouble controlling my anger.</mark>

- <mark>k) I wonder why sometimes I feel so bitter about things.</mark>

<mark>l) I sometimes feel like a powder keg ready to explode.</mark>

# **<mark>Risk for mental health problems</mark>**

# _<mark>English text of Facebook questionnaire</mark>_

|26<br>We would now like to ask you some questions about<br>your general health over the past few weeks.<br>Response options: 1. Much better than usual 2. Better<br>than usual 3. No difference/as usual 4. Worse than<br>usual 5. Much worse than usual|
|---|
|Canyou focus onyour work as usual?|
|Do you feel you are playing a useful role toward the<br>people aroundyou?|
|Canyou make decisions as usual?|
|Doyou feel able to faceyourproblems?|
|Are you happy and satisfied with your<br>accomplishment at work?|
|Are you able to feel happy notwithstanding the<br>surrounding circumstances?|

# _<mark>Notes on comparability</mark>_

The PPCS intersperses positive and negative items with a single four-point response scale from better than usual to much worse than usual. Due to confusion in the development process, the Facebook questionnaire maintained two separate response scales for positive and negative items. The items were therefore presented in two banks of questions to minimize space in the online survey format.

While the Facebook questionnaire asks about the past few weeks, the PPCS questionnaire refers to the past two weeks.

19
28 We would like to follow up with some more questions about your general health over the last few weeks. Response options: 1. No never 2. Not more than usual 3. More than usual 4. Much more than usual Do you find it difficult to sleep because you are nervous or preoccupied? <mark>Do you feel under continuous pressure? Do you feel capable of overcoming</mark> <u><mark>your problems?</mark></u> <mark>Do you feel sad and that there is no way out? Have you lost your self-confidence?</mark> The question “Do you feel capable of overcoming your problems?” is negatively phrased in the PPCS (“Do you feel incapable of overcoming your problems?”). In analysis of the Facebook data, this item has been reverse coded so the higher end of the response scale indicates more severe outcomes, in alignment with the rest of the module.

<mark>Do you see yourself as a useless person?</mark>

# **<mark>Depression</mark>**

_<mark>English text of Facebook questionnaire</mark>_ 29 In the past two weeks, how often were the following statements true for you? Response options: 0. Never 1. A little of the time (rarely) 2. Slightly less than half the time 3. More than half of the time 4. More often 5. Always <mark>You were happy with a good mood You were feeling calm and relaxed You were feeling energetic You woke up active and relaxed Your days were full of things you liked</mark> _<mark>Notes on comparability</mark>_

|**C**<br>_English text of Facebookquestionnair_|**OVID-19 exp**<br>_e_|**osure**<br>_Notes on comparability_|
|---|---|---|
|31<br>Have you ever had, or do you believe that<br>you have ever had Coronavirus?|Yes / No / I<br>don’t<br>know|In the PPCS, the first question in this module offers<br>only Yes / No response options.|
|In the last 14 days, do you think you have<br>met (seen) anyone who you think had the<br>coronavirus whenyou met them?|Yes / No / I<br>don’t<br>know||
|Do you think your area has a high incidence<br>of coronavirus?|Yes / No / I<br>don’t<br>know||
|Do you personally know someone who has<br>died from the coronavirus in your area?|Yes / No / I<br>don’t<br>know||

20
# Appendix B: Item-level sensitive response shares analysis

## **Personal exposure to conflict events**

|**Question**|**Share of r**<br>**choosing**<br>**that indi**<br>**severe ou**<br>_PPCS_|**espondents**<br>**responses**<br>**cate more**<br>**tcomes (%)**<br>_Facebook_|**Difference**<br>**(Percentage**<br>**point)**|<br>**N**<br>**(combined)**|
|---|---|---|---|---|
|Have you been injured due to Israeli airstrikes or|1.01|9.29|8.28|2929|
|clashes with Israeli security forces in May 2021?|(0.20)|(1.52)|(1.53)||
|Has a household member of yours died due to|0.23|6.22|5.98|2933|
|Israeli airstrikes or because of clashes with Israeli<br>security forces in May 2021?|(0.09)|(1.26)|(1.26)||
|Was any other household member injured in|1.37|11.26|9.90|2927|
|Israeli airstrikes or clashes with Israeli security|(0.27)|(1.66)|(1.68)||
|forces in May 2021?|||||
|Did an extended family member or close friend die|10.53|32.08|21.54|2934|
|due to Israeli airstrikes or clashes with Israeli<br>security forces in May 2021?|(1.16)|(2.43)|(2.69)||
|Was an extended family member or close friend|10.22|38.04|27.82|2931|
|injured in Israeli airstrikes or clashes with Israeli|(0.90)|(2.53)|(2.69)||
|security forces in May 2021?|||||
|Was your workplace demolished or damaged due|2.26|35.91|33.64|2900|
|to Israeli airstrikes in May 2021?|(0.32)|(2.62)|(2.64)||
|Was your home destroyed or damaged due to|26.65|36.39|9.74|2923|
|Israeli airstrikes in May 2021?|(1.93)|(2.54)|(3.19)||
|_Average _|7.47|24.17|16.70|2925|

Notes: Responses indicating more severe outcomes are operationalized in this module as responses of “Yes”. Contingent questions (following up on the answers to the last two questions) have been excluded. Estimates are unweighted, and clustered standard errors are reported in parentheses.

|**A**<br>**Statement**|**ggression**<br>**Share of r**<br>**choosing**<br>**that indi**<br>**severe ou**<br>_PPCS_|**espondents**<br>**responses**<br>**cate more**<br>**tcomes (%)**<br>_Facebook_|**Difference**<br>**(Percentage**<br>**point)**|**N**<br>**(combined)**|
|---|---|---|---|---|
|Given enough provocation, I may hit another|16.64|16.68|0.04|6859|
|person.|(0.84)|(1.19)|(1.46)||
|I often find myself disagreeing with people.|24.63|40.12|15.50|6853|
||(1.06)|(1.57)|(1.89)||
||28.83|59.28|30.45|6851|

21
|At times I feel I have gotten a raw deal out of|||||
|---|---|---|---|---|
|life.|(1.12)|(1.57)|(1.93)||
|There are people who have pushed me so far<br>|9.46|24.46|15.00|6849|
|that we have come to blows.|(0.53)|(1.38)|(1.48)||
|I can't help getting into arguments when|21.09|31.31|10.22|6847|
|people disagree with me.|(0.88)|(1.49)|(1.73)||
|Sometimes I fly off the handle for no good|28.05|27.26|-0.78|6848|
|reason.|(1.02)|(1.43)|(1.76)||
|Other people always seem to get the breaks.|21.89|44.55|22.66|6848|
||(0.95)|(1.59)|(1.86)||
|I have threatened people I know.|4.54|6.42|1.88|6841|
||(0.41)|(0.79)|(0.89)||
|My friends say I am somewhat|11.91|22.71|10.80|6849|
|argumentative.|(0.75)|(1.34)|(1.54)||
|I have trouble controlling my anger.|25.99|29.39|3.40|6839|
||(1.00)|(1.47)|(1.78)||
|I wonder why sometimes I feel so bitter|26.68|44.13|17.45|6839|
|about things.|(1.13)|(1.60)|(1.96)||
|I sometimes feel like a powder keg ready to|26.16|51.91|25.75|6845|
|explode.|(1.15)|(1.61)|(1.97)||
|_Average _|20.49|33.19|12.70|6847|

Notes: Responses indicating more severe outcomes are operationalized in this module as responses of “somewhat like me” and “very like me”. Estimates are unweighted, and clustered standard errors are reported in parentheses.

|**Risk for me**<br>**Question**|**ntal healthpr **<br>**Share of r**<br>**choosing**<br>**that indi**<br>**severe ou**<br>_PPCS_|**oblems**<br>**espondents**<br>**responses**<br>**cate more**<br>**tcomes (%)**<br>_Facebook_|**Difference**<br>**(Percentage**<br>**point)**|**N**<br>**(combined)**|
|---|---|---|---|---|
|Can you focus on your work as usual?|20.34|31.17|10.84|6848|
||(0.80)|(1.49)|(1.69)||
|Do you find it difficult to sleep because you|32.03|47.36|15.33|6862|
|are nervous or preoccupied?|(1.04)|(1.59)|(1.90)||
|Do you feel you are playing a useful role|8.66|10.92|2.26|6856|
|toward the people around you?|(0.49)|(1.00)|(1.11)||
|Can you make decisions as usual?|10.30|18.50|8.20|6849|
||(0.52)|(1.25)|(1.35)||
|Do you feel under continuous pressure?|29.59|56.47|26.87|6850|
||(1.01)|(1.59)|(1.88)||
|Do you feel incapable of overcoming your|16.59|68.14|51.54|6852|
|problems?|(0.74)|(1.49)|(1.66)||
||10.69|29.48|18.80|6846|

22
|Are you happy and satisfied with your<br>accomplishment at work?|(0.53)|(1.46)|(1.56)||
|---|---|---|---|---|
|Do you feel able to face your problems?|13.29|23.75|10.45|6853|
||(0.65)|(1.36)|(1.51)||
|Do you feel sad and that there is no way out?|23.01|48.31|25.30|6851|
||(0.82)|(1.60)|(1.80)||
|Have you lost your self-confidence?|7.88|22.42|14.54|6853|
||(0.46)|(1.33)|(1.41)||
|Do you see yourself as a useless person?|7.13|16.39|9.26|6852|
||(0.43)|(1.19)|(1.26)||
|Are you able to feel happy notwithstanding|11.11|35.06|23.94|6863|
|the surrounding circumstances?|(0.56)|(1.52)|(1.62)||
|_Average _|15.89|34.00|18.11|6853|

Notes: Responses indicating more severe outcomes are operationalized in this module as responses of “worse than usual”/“much worse than usual” or “more than usual”/“much more than usual”. Estimates are unweighted, and clustered standard errors are reported in parentheses.

|**Statement**|**Depression**<br>**Share of r**<br>**choosing**<br>**that indi**<br>**severe ou**<br>_PPCS_|**espondents**<br>**responses**<br>**cate more**<br>**tcomes (%)**<br>_Facebook_|**Difference**<br>**(Percentage**<br>**point)**|**N**<br>**(combined)**|
|---|---|---|---|---|
|You were happy with a good mood|62.41|66.19|3.78|6855|
||(1.21)|(1.51)|(1.94)||
|You were feeling calm and relaxed|64.84|74.26|9.42|6855|
||(1.12)|(1.40)|(1.79)||
|You were feeling energetic|59.05|65.26|6.21|6849|
||(1.15)|(1.53)|(1.91)||
|You woke up active and relaxed|56.94|67.69|10.74|6857|
||(1.16)|(1.49)|(1.89)||
|Your days were full of things you liked|62.78|72.90|10.12|6854|
||(1.21)|(1.42)|(1.87)||
|_Average _|61.20|69.26|8.06|6854|

Notes: Responses indicating more severe outcomes are operationalized in this module as responses of “never”, “rarely”, and “slightly less than half the time”. Estimates are unweighted, and clustered standard errors are reported in parentheses.

23
# **Food Insecurity Experience Scale (FIES)**

|**Statement**|**Share of**<br>**choosin**<br>**that ind**<br>**severe o**<br>_PPCS_|**respondents**<br>**g responses**<br>**icate more**<br>**utcomes (%)**<br>_Facebook_|**Difference**<br>**(Percentage**<br>**point)**|**N**<br>**(combined)**|
|---|---|---|---|---|
|You worried that the household would not|42.07|56.67|14.60|5775|
|have enough food|(1.68)|(2.18)|(2.75)||
|You or any household members were not able|43.00|62.36|19.36|5784|
|to eat the kinds of foods you prefer because of|(1.74)|(2.11)|(2.74)||
|a lack of resources|||||
|You or any household member ate a limited|42.71|60.59|17.89|5797|
|variety of food because of a lack of resources|(1.79)|(2.11)|(2.76)||
|You or any household member ate a smaller|31.16|56.11|24.95|5800|
|meal than you felt you needed because there<br>was not enough food|(1.51)|(2.14)|(2.62)||
|You or any household member ate fewer meals|26.96|53.85|26.88|5792|
|a day because there was not enough food|(1.36)|(2.16)|(2.55)||
|There was no food at all in your household|19.77|36.28|16.51|5788|
|because there were no resources to get more|(1.06)|(2.09)|(2.34)||
|You or any household member go to sleep at|8.97|35.94|26.97|5797|
|night hungry because there was not enough|(0.68)|(2.07)|(2.18)||
|food|||||
|You or any household member went for a|6.35|25.98|19.63|5795|
|whole day and night without eating because|(0.58)|(1.90)|(1.98)||
|there was not enough food|||||
|_Average _|34.28|54.31|20.03|5789|

Notes: Responses indicating more severe outcomes are operationalized in this module as responses of “once or twice”, “3 to 10 times”, and “more than 10 times”. Estimates are unweighted, and clustered standard errors are reported in parentheses.

# **Reduced Coping Strategy Index (RCSI)**

|**Statement**|**Share of r**<br>**choosing**<br>**that indi**<br>**severe ou**<br>_PPCS_|**espondents**<br>**responses**<br>**cate more**<br>**tcomes (%)**<br>_Facebook_|**Difference**<br>**(Percentage**<br>**point)**|**N**<br>**(combined)**|
|---|---|---|---|---|
|Reduced the number of meals for all|27.64|63.59|35.95|6450|
|household members per day|(1.31)|(2.01)|(2.40)||
|Reduced the quantity of meals eaten by adults|25.09|56.45|31.37|6434|
|in favor of children|(1.24)|(2.10)|(2.44)||
|Purchased low quality markets "Leftover"|35.99|46.21|10.22|6430|

24
||(1.51)|(2.12)|(2.60)||
|---|---|---|---|---|
|Reduced portion of food for adults in favor of|27.03|51.72|24.69|6429|
|children's|(1.32)|(2.13)|(2.50)||
|borrow food or rely on help from family and|21.85|39.05|17.20|6424|
|friends|(1.10)|(2.08)|(2.36)||
|Refrain from consuming expensive and resort|42.51|75.71|33.20|6436|
|to alternatives (buying cheaper kind of food)|(1.61)|(1.81)|(2.42)||
|_Average _|30.02|55.46|25.44|6434|

Notes: Responses indicating more severe outcomes are operationalized in this module as responses of 1-7 days. Estimates are unweighted, and clustered standard errors are reported in parentheses.

|**COVID**<br>**Question**|**-19 exposur**<br>**Share of r**<br>**choosing**<br>**that indi**<br>**severe ou**<br>_PPCS_|**e**<br>**espondents**<br>**responses**<br>**cate more**<br>**tcomes (%)**<br>_Facebook_|**Difference**<br>**(Percentage**<br>**point)**|**N**<br>**(combined)**|
|---|---|---|---|---|
|Have you ever had, or do you believe that you|46.32|48.60|2.28|6804|
|have ever had Coronavirus?|(1.04)|(1.64)|(1.94)||
|In the last 14 days, do you think you have met|23.66|36.67|13.01|6806|
|(seen) anyone who you think had the<br>coronavirus when you met them?|(1.11)|(1.58)|(1.93)||
|Do you think your area has a high incidence of|28.54|23.93|-4.61|6808|
|coronavirus?|(1.35)|(1.40)|(1.94)||
|Do you personally know someone who has|55.24|75.27|20.03|6802|
|died from the coronavirus in your area?|(1.33)|(1.42)|(1.94)||
|_Average _|38.44|46.12|7.68|6805|

Notes: Responses indicating more severe outcomes are operationalized in this module as responses of “Yes”. Estimates are unweighted, and clustered standard errors are reported in parentheses.

25
<!-- Start of picture text --> Shareto atofleastonele who conflicteventrt personal (Gazaexposure only) ApproximatePr or severeprevalence food insecurityof moderate<br>4 36<br>35 3<br>25 2 4<br>Vr A&<br>© Unrestricted sample © Fiche education © Unrestricted sample © SecondatyoF<br>@ Men ® Workedin past week ® Men ® Workedin past week<br>Average Reduced Coping Strategies Index Average aggressive tendencies index<br>10: 8<br>86<br>75<br>6+ 4<br>@ Unrestricted sample © Fgh ocktcatsi ®@ Unvrestrctedsampie ence a<br>® Men ® Worked in past week @ Men ® Worked in past week<br>Average risk for mental health problems index Prevalence of symptoms consistent with depression<br>25 2<br>48<br>152 | + |<br>05<br>4 i]<br>@ Unrestricted sample © fiohier edcalich ® Unrestrctedsampie © oho ecw<br>® Men ® Workedin past week ® Men ® Workedin past week<br>Average depression symptom severity level Share of ple who report at least<br>10 7 one cecal omenee te COVID-19<br>14<br>8 12<br>6 1 ?<br>‘ 08 |<br>06<br>2 04<br>® Unrestricted sample @ crt A © Unrestricted sample @ Fah alcation<br>@ Men ® Workedin past week © Men ® Workedin past week<br><!-- End of picture text -->
men, and people who reported working in the past week. All estimates are unweighted, and 95% confidence intervals are based on clustered standard errors.

27