Online misinformation is linked to early COVID-19 vaccination hesitancy and refusal

<jats:title>Abstract</jats:title><jats:p>Widespread uptake of vaccines is necessary to achieve herd immunity. However, uptake rates have varied across U.S. states during the first six months of the COVID-19 vaccination program. Misbeliefs may play an important role in vaccine hesitancy, and there is a need to understand relationships between misinformation, beliefs, behaviors, and health outcomes. Here we investigate the extent to which COVID-19 vaccination rates and vaccine hesitancy are associated with levels of online misinformation about vaccines. We also look for evidence of directionality from online misinformation to vaccine hesitancy. We find a negative relationship between misinformation and vaccination uptake rates. Online misinformation is also correlated with vaccine hesitancy rates taken from survey data. Associations between vaccine outcomes and misinformation remain significant when accounting for political as well as demographic and socioeconomic factors. While vaccine hesitancy is strongly associated with Republican vote share, we observe that the effect of online misinformation on hesitancy is strongest across Democratic rather than Republican counties. Granger causality analysis shows evidence for a directional relationship from online misinformation to vaccine hesitancy. Our results support a need for interventions that address misbeliefs, allowing individuals to make better-informed health decisions.</jats:p>

section_content: Here we study relationships between vaccine uptake, vaccine hesitancy, and online misinformation. Leveraging data from Twitter, Facebook, and the Centers for Disease Control and Prevention (CDC), we investigate how online misinformation is associated with vaccination rates and levels of vaccine hesitancy across the U. S. We also use Granger Causality analysis to investigate whether there is evidence for a directional association between misinformation and vaccine hesitancy.

section_content: Our key independent variable is the mean percentage of vaccine-related misinformation shared via Twitter at the U. S. state or county level. We used 55 M tweets from the CoVaxxy dataset 17, which were collected between January 4th and March 25th from the Twitter filtered stream API using a comprehensive list of keywords related to vaccines (see Supplementary Information). We leveraged the Carmen library 29 to geolocate almost 1. 67 M users residing in 50 U. S. states, and a subset of approximately 1. 15 M users residing in over 1,300 counties. The larger set of users accounts for a total of 11 M shared tweets. Following a consolidated approach in the literature [25] [26] [27] [28], we identified misinformation by considering tweets that contained links to news articles from a list of lowcredibility websites compiled by a politically neutral third party (see details in the Supplementary Information). We measured the prevalence of misinformation about vaccines in each region by (i) calculating the proportion of vaccine-related misinformation tweets shared by each geo-located account; and (ii) taking the average of this proportion across accounts within a specific region. The Twitter data collection was evaluated and deemed exempt from review by the Indiana University IRB (protocol 1102004860). 

Our dependent variables include vaccination uptake rates at the state level and vaccine hesitancy at the state and county levels. Vaccination uptake is measured from the number of daily vaccinations administered in each state during the week of 19-25 March 2021, and measurements are derived from the CDC 9. Vaccine hesitancy rates are based on Facebook Symptom Surveys provided by the Delphi Group 24 at Carnegie Mellon University. Vaccine hesitancy is likely to affect uptake rates, so we specify a longer time window to measure this variable, i. e., the period January 4th-March 25th 2021. We computed hesitancy by inverting the proportion of individuals "who either have already received a COVID vaccine or would definitely or probably choose to get vaccinated, if a vaccine were offered to them today. " See Supplementary Information for further details. 

There are no missing vaccine-hesitancy survey data at the state level. Observations are missing at the county level because Facebook survey data are available only when the number of respondents is at least 100. We use the same threshold on the minimum number of Twitter accounts geolocated in each county, resulting in a sample size of N = 548 counties. 

Our multivariate regression models adjust for six potential confounding factors: percentage of the population below the poverty line, percentage aged 65 +, percentage of residents in each racial and ethnic group (Asian, Black, Native American, and Hispanic; White non-Hispanic is omitted), rural-urban continuum code (RUCC, county level only), number of COVID-19 deaths per thousand, and percentage Republican vote (in 10 percent units). Other covariates, including religiosity, unemployment rate, and population density, were also considered (full list in Supplementary Table S9 ). 

We also conduct a large number of sensitivity analyses, including different specifications of the misinformation variable (with a restricted set of keywords and different thresholds for the inclusion of Twitter accounts) as well as logged versions of misinformation (to correct positive skew). These results are presented in Supplementary Information (Tables S3-S8 ). 

We conduct multiple regression models predicting vaccination rate and vaccine hesitancy. Both dependent variables are normally distributed, making weighted least squares regression the appropriate model. Data are observed (aggregated) at the state or county level rather than at the individual level. Analytic weights are applied to give more influence to observations calculated over larger samples. The weights are inversely proportional to the variance of an observation such that the variance of the j-th observation is assumed to be σ 2 /w j where w j is the weight. The weights are set equal to the size of the sample from which the average is calculated. We estimate weighted regression with the aweights command in Stata 16. In addition, because counties are nested hierarchically within states, we use cluster robust standard errors to correct for lack of independence between county-level observations. We investigate Granger causality between vaccine hesitancy and misinformation by comparing two autoregressive models. The first considers daily vaccine hesitancy rates x at time t in geographical region r (state or county):

where n is the length of the time window. The second model adds daily misinformation rates per account as an exogenous variable y:

The variable y is said to be Granger causal 30, 31 on x if, in statistically significant terms, it reduces the error term ǫ ′ t, i. e., if

meaning that misinformation rates y help forecast hesitancy rates x. We assume geographical regions to have equivalence and independence in terms of the way misinformation influences vaccine attitudes. Thus, we use the same parameters for a i and b i across all regions. We employ Ordinary Least Squares (using the Python statsmodels package version 0. 11. 1) linear regression to fit a and b, standardizing the two variables and removing trends in the time series of each region. We select the value of the time window n that maximizes E a,b. For both counties and states, this was n = 6 days and we present results using this value. We also tested nearby values of n ± 2 to confirm these provide similar results. We use data points with at least 1 tweet and at least 100 survey responses for every day in the time window for the specified region. The traditional statistic used to assess the significance of Granger Causality is the F-statistic 30. However, in our case, there are several reasons why this is not appropriate. First, we have missing time windows in some of our regions. Second, our assumptions of equivalence and independence for regions may not be accurate. For these reasons, we use a bootstrap method to estimate the expected random distribution of E a,b with the time signal removed. To this end, we generate trial surrogates for y by randomly shuffling the data points. With each random reshuffled trial, we can then use the same procedure to calculate the reduction in error, which we call E * a,b. The p-value of our Granger Causality analysis is then given by the proportion of trials ( N = 10,000) for which E * a,b > E a,b. A potential issue with Granger Causality analysis is that it may detect an underlying trend. We tested for this by linearly detrending both time series before running the Granger analysis, finding similar results.
section_name: Methods

section_content: Looking across U. S. states, we observe a negative association between vaccination uptake rates and online misinformation (Pearson R = -0. 49, p < 0. 001). Investigating covariates known to be associated with vaccine uptake or hesitancy, we find that an increase in the mean amount of online misinformation is significantly associated with a decrease in daily vaccination rates per million (b = -3518. 00, p = 0. 009, Fig. 1A, and see Methods and Supplementary Table S1 ). Political partisanship (a 10% increase in GOP vote) is also strongly associated with vaccination rate (b = -640. 32, p = 0. 004). These two factors alone explain nearly half the variation in state-level vaccination rates, and are themselves moderately correlated (Supplementary Fig. S1 and Table S1 ), consistent with prior research 32. Remaining covariates are non-significant and/or collinear with other variables (i. e., have high variance inflation factors) and thus dropped for parsimony. 

To investigate vaccine hesitancy, we leverage over 22 M individual responses to daily survey data provided by Facebook 24 (see Methods). Reports of vaccine hesitancy are aggregated at the state level (given as a percentage) and weighted by sample size. We find a strong negative correlation between vaccine uptake and hesitancy across U. S. states (Pearson R = -0. 71, p < 0. 001, Supplementary Fig. S1 ), suggesting that daily vaccination rates largely reflect demand for vaccines rather than supply. Taking into account the same set of potential confounding factors in a weighted regression model, we find a significant positive association between misinformation and state-level vaccine hesitancy (b = 6. 88, p = 0. 007), and between political partisanship and hesitancy (b = 2. 96, p < 0. 001; see Fig. 1B and Supplementary Fig. S1 ). Figure 1C illustrates the state-level correlation between misinformation and hesitancy. For example, the large size and yellow color of Wyoming indicate it is the state with the highest level of misinformation and hesitancy. Among other variables, we find that the percentage of Black residents is positively related to reports of hesitancy (b = 0. 12, p = 0. 001), while the percentage of Hispanic or Latinx residents is negatively associated (b = -0. 07, p = 0. 021). The percentage of residents below the poverty line is also positively associated with vaccine hesitancy (b = 0. 53, p = 0. 001). 

To test the robustness of these results, we also consider a more granular level of information by examining county data. Similar to previous analyses, we compute online misinformation shared by almost 1. 15 M Twitter users geolocated in over 1,300 U. S. counties. We measure vaccine hesitancy rates by leveraging over 17 M daily responses to the Facebook survey for over 700 distinct counties. The total number of observations (counties) for which we are able to measure both variables is N = 548 (see Methods). Political partisanship and misinformation are both significantly correlated with county-level vaccine hesitancy, net covariates (Supplementary Table S4 and Supplementary Fig. S2 ). Using a weighted multiple linear regression model, we find a significant interaction between political partisanship and misinformation. Specifically, as levels of misinformation increase, Democratic and Republican counties converge to the same level of vaccine hesitancy (Fig. 2 ). This may suggest the presence of a ceiling effect at around 30% of residents being vaccine hesitant (on average), with Republican counties having already reached the ceiling and thus their residents being less likely to be affected by misinformation. 

Our results so far demonstrate an association between online misinformation and vaccine hesitancy. We investigate evidence for directionality in this association by performing a Granger Causality analysis 30, 31. We find that misinformation helps forecast vaccine hesitancy, weakly at state level (p = 0. 0519) and strongly at county level (p < 0. 001; see Methods and Supplementary Tables S10, S11 ). Analysis of the significant lagged coefficients (Supplementary Table S10 ) indicates that there is a lag of around 2-6 days from misinformation posted in a county to a corresponding increase in vaccine hesitancy in the same county. Finally, Fig. 3 shows the most shared low-credibility sources. We note the large prevalence of one particular source, Children's Health Defense, an anti-vaccination organization that has been identified as one of the main sources of misinformation on vaccines 34, 35. We did not observe significant differences in the top sources shared in Republican vs. Democratic majority states.
section_name: Results

section_content: Our results provide evidence for the problem of geographical regions with lower levels of COVID-19 vaccine uptake, which may be driven by online misinformation. Considering variability across regions with low and high levels of misinformation, the best estimates from our data predict a ~ 20% decrease in vaccine uptake between states, and a ~ 67% increase in hesitancy rates across Democratic counties, across the full range of misinformation prevalence. At these levels of vaccine uptake, the data predict SARS-CoV-2 will remain endemic in many U. S. regions. This suggests a need to counter misinformation in order to promote vaccine uptake. 

An important question is whether online misinformation drives vaccine hesitancy. Our analyses alone do not demonstrate a causal relationship between misinformation and vaccine refusal. Our work is at an ecological scale and vaccine-hesitant individuals are potentially more likely to post vaccine misinformation. However, at the individual level, a recent study 20 found that exposure to online misinformation can increase vaccine hesitancy. Our work serves to provide evidence that those findings, which were obtained under controlled circumstances, scale to an ecological setting. Due to the fact that vaccine hesitancy and misinformation are socially reinforced, both ecological and individual relationships are important in demonstrating a causal link 36. However, we are still unable to rule out confounding factors, so uncertainty remains about a causal link and further investigation is warranted. 

Public opinion is very sensitive to the information ecosystem and sensational posts tend to spread widely and quickly 25. Our results indicate that there is a geographical component to this spread, with opinions on vaccines spreading at a local scale. While social media users are not representative of the general public, existing evidence suggests that vaccine hesitancy flows across social networks 37, providing a mechanism for the lateral spread of misinformation offline among those connected directly or indirectly to misinformation spreading online. More broadly, our results provide additional insight into the effects of information diffusion on human behavior and the spread of infectious diseases 38. 

A limitation of our findings is that we are not measuring the exposure, by geographical region, to misinformation on Twitter but rather the sharing activity of a subset of users. Besides, our analyses are based on data averaged over geographical regions. To account for group-level effects we present a number of sensitivity analyses, and note that our findings are consistent over two geographical scales. Our source-based approach to detect misinformation at scale might not capture the totality of misleading and harmful content related to vaccines, and many low-credibility sources publish a mixture of false and true information 39, 40. Our results are also limited to a small period of time. Finally, other factors might also influence vaccination hesitancy levels, including accessibility to vaccines, changes in COVID-19 infection and death rates, as well as legitimate reports about vaccine safety 41. 

Associations between online misinformation and detrimental offline effects, like the results presented here, call for better moderation of our information ecosystem. COVID-19 misinformation is shared overtly by known entities on major social media platforms 42. While people have a constitutional right to free speech, it is important to maintain an environment where individuals have access to good information that benefits public health. 

Data and code availability. All measurements of vaccine uptake and vaccine hesitancy rates as well as socioeconomic, political, and demographic variables at the state and county level are publicly available in the online repository associated with this paper 43. We also provide aggregated measurements of online misinformation shared by geolocated Twitter users. Results at the state and county level can be fully reproduced using the STATA scripts provided in the repository. Due to Twitter's terms of use and service, we can only release IDs of the tweets present in our dataset, which can be reconstructed using the Twitter API. The IDs are accessible in the public dataset associated with the CoVaxxy project 17 from the Observatory on Social Media at Indiana University.
section_name: Discussion

section_name: Acknowledgements

section_content: We are grateful to Filipi Silva for help with the Granger Causality analysis; John Bollenbacher, Bao Tran Truong, David Axelrod, Niklas Loynes, and Christopher Torres-Lugo for their contributions to the CoVaxxy data collection and for helpful discussion; to George Turner for help with the XSEDE computer platform; and to JangDong Seo, Edwin van Leeuwen, and Francesco Scotti for insightful discussions on statistical analyses. This work is supported in part by Knight Foundation, Craig Newmark Philanthropies, the National Science Foundation (grant ACI-1548562 ), and EU H2020 Periscope (grant 101016233 ).
section_name: Acknowledgements

section_name: Author contributions

section_content: J. B., F. P., and F. M. designed the study. F. P., M. D., K. Y., and J. B. collected and processed the data. B. P., F. P., and J. B. performed the analyses. J. B., F. P., and B. P. wrote the manuscript with contributions from all authors. J. B., A. F., and F. M. directed the project.
section_name: Author contributions

section_content: The authors declare no competing interests.
section_name: Competing interests