# **1. Introduction**

Can Twitter data be used to monitor attitudes towards COVID-19 vaccination? How do estimates of vaccination hesitancy rates derived from Twitter data compare to estimates obtained from more conventional survey data? Who are the most effective messengers on Twitter and what are the most effective messages used on either side of the discourse (supporters versus opponents of COVID vaccination)?

We investigate these questions in an application to the Middle East for which we have both conventional survey data on COVID vaccination attitudes and a database of tweets that feature the most prominent COVID hashtag in Arabic spanning the period from mid-December 2020 until mid-April 2021. To identify a given Twitter account’s attitude towards COVID vaccination (for all accounts observed in our database), we adopt the following approach. First, we identify the clusters of user accounts observed in our data. (Clusters are groups of users who are more prone to interacting with each other on Twitter. The resulting clusters, while purely identified by exploiting data on retweet behavior, tend be divided across geographic and ideological lines.) Second, we identify the 50-100 most influential accounts from each cluster. These influencers represent the central nodes in the network of Twitter accounts. Third, we manually inspect the Twitter feeds of these influencers for explicit references towards COVID vaccination that allows us to label them as either pro-vaccination or anti-vaccination (or neutral with respect to COVID vaccination). Fourth, and finally, we iteratively impute the vaccination attitude of other users in our database (i.e. who participated in the discourse on COVID vaccination on Twitter) based on their retweeting of users whom vaccination attitude has been established. We hereby rely on the assumption that a retweet without the addition of any comments is interpreted as an endorsement.

We find that COVID vaccination hesitancy on Twitter in the Middle East is estimated at just below 20 percent, which is of a comparable magnitude as hesitancy rates estimated from survey data, provided that the appropriate filters are applied. At first glance, Twitter-based estimates of hesitancy appear notably downwards biased; when anti-vaccine attitudes are evaluated over the full population of Twitter accounts who engaged in the discourse on COVID vaccination, hesitancy is estimated at around 3 percent. This bias is fully accounted for, however, when we: (1) evaluate hesitancy over Twitter accounts belonging to individuals (i.e. filtering out accounts belonging to organizations who make up a significant share of the discourse on Twitter), and (2) adjust for the fact that the population of Twitter users is biased towards more educated individuals.

With regards to effective messengers and messages in the discourse on COVID vaccination, we find that the most effective tweets on the anti-vaccine side highlight claims that the vaccine causes serious life-threatening side effects -- while on the pro-vaccine side, tweets showing public figures receiving the vaccine are found to be most effective. Finally, state-news accounts are found to receive the overwhelming majority of retweets in our database making them the most effective messengers.

2