The phone survey asked a series of questions about the potential drivers of COVID-19 vaccine hesitancy along with questions about the level of hesitancy (see Appendix B). High levels of vaccine hesitancy were anticipated because a prior phone survey had captured preliminary data about willingness to be vaccinated in the event that an approved vaccine became available at no cost. As such the focus of the phone survey drawn on in this paper was more on understanding what was driving the high levels of hesitancy as opposed to examining the prevalence of vaccine hesitancy in PNG. To achieve this aim, an array of questions were included allowing for a detailed examination of the extent to which different factors influence vaccine hesitancy. These questions were sourced directly from a combination of COVID-19 vaccine hesitancy surveys that the World Bank, UNICEF and WHO had undertaken in PNG or in other low- and middle-income countries. The phone survey went to the field as the national rollout of the vaccine was beginning and as a result the vaccine questions were asked assuming respondents could potentially have already received their first dose.

The data collected from the phone survey were analyzed by identifying correlations and conducting regression analysis. The descriptive findings were calculated using household weights derived from the 2016-18 DHS to ensure the overall findings were broadly nationally representative, and throughout the paper household-weighted findings are reported on unless otherwise stated. The regression analysis involved using ordinary least squares (OLS) to estimate the effect on willingness to be vaccinated (the dependent variable) from a dummy variable for a given potential driver of vaccine hesitancy (the independent variable) after controlling for demographic characteristics of respondents.

# <u>4.2 Online randomized survey experiment methodology</u>

The online randomized survey experiment was conducted via Facebook from June 24 to July 26, 2021. Participants were recruited using Facebook advertisements and taken to a chatbot in Facebook messenger to complete the survey. The sample was drawn from all Facebook accounts in PNG and was stratified by demographic groups (based on age, gender and regions) to ensure adequate representation in the sample. There were almost one million social media users in PNG in January 2021 – over 10 percent of the entire population or almost 20 percent of the adult population (that is, those aged 18 years or older) (Hoodsuite, 2021). The survey questions measured basic demographics, vaccine intentions/take-up and behavioral dimensions such as social norms, trust, and role models (see Appendix B). This allowed for an estimate of the overall levels of intention to be vaccinated. Although the online experiment itself was not repeated in the phone survey, all of the questions in the Facebook experiment were also included in the phone survey.

The treatments in the online experiment were based on questions trialed in other settings and were also informed by the results of the phone survey. A series of Facebook experiments about vaccine hesitancy were run by the World Bank in the first half of 2021 in the Middle East and North Africa using a range of behaviorally informed messages. These treatments were the starting point for this study. However, questions were tailored to suit the PNG context based on extensive consultation with stakeholders. In addition, treatments were only included in the experiment if the findings of the phone survey suggested there was a positive correlation between a potential driver of vaccine hesitancy and people’s willingness to be vaccinated. This resulted of the three treatments (expert advice, social norms and relative safety) are shown below (see Figure 3).

# **Figure 3 – Design of randomized survey experiment**

11