Revealing COVID -19’s Social Dynamics: Diachronic Semantic Analysis of Vaccine and Symptom Discourse on T witter (2024)

Abstract: Social media is recognized as an important source for deriving insights into public opinion dynamics and social impacts due to the vast textual data generated daily and the ‘unconstrained’ behavior of people interacting on these platforms. However, such analyses prove challenging due to the semantic shift phenomenon, where word meanings evolve over time. This paper proposes an unsupervised dynamic word embedding method to capture longitudinal semantic shifts in social media data without predefined anchor words. The method leverages word co-occurrence statistics and dynamic updating to adapt embeddings over time, addressing the challenges of data sparseness, imbalanced distributions, and synergistic semantic effects. Evaluated on a large COVID-19 Twitter dataset, the method reveals semantic evolution patterns of vaccine- and symptom-related entities across different pandemic stages, and their potential correlations with real-world statistics. Our key contributions include the dynamic embedding technique, empirical analysis of COVID-19 semantic shifts, and discussions on enhancing semantic shift modeling for computational social science research. This study enables capturing longitudinal semantic dynamics on social media to understand public discourse and collective phenomena.

## introduction social media has become an essential platform for information dissemination and public opinion expression, with over 4.5 billion global users generating vast amounts of textual data daily @xcite @xcite . large-scale, wide-coverage, and real-time social media data provide a new source for computational social science and cultural analysis research, enabling insights into public opinion dynamics, so-cial event impacts, and collective behavior patterns through large-scale text analysis @xcite . however, social media text analysis faces a unique challenge: the semantic shift phenomenon. word semantics can change to a certain extent over time due to evolving social events, user interests, cultural trends, and other factors @xcite . for instance, during the covid-19 pandemic, "moderna" initially referred to a biotechnology company but later became associated with its covid-19 vaccine. failing to account for such dynamic semantic evolution can affect the accuracy and interpretability of text analysis, especially in long-term longitudinal analyses spanning months or even years. existing mainstream semantic shift analysis methods usually require pre-defining a set of anchor words as a reference @xcite . however, these methods rely excessively on human experts' prior experiences that are usually highly subjective @xcite , and cannot comprehensively detect semantic shifts in longitudinal corpora. moreover, social media text presents several challenges for existing semantic shift analysis methods: to address these challenges, this paper proposes a novel framework aimed at better capturing longitudinal semantic evolution patterns in the context of social media and providing powerful semantic analysis tools for computational social science research. the proposed end-to-end unsupervised dynamic word embedding method is based on word co-occurrence statistics and combines a dynamic update strategy to adaptively capture the trajectory of word semantic evolution over time, without the need for manually defined anchor words. to verify the effectiveness of the method in practical applications, we applied it to a large-scale covid-19 twitter dataset spanning more than two years, from february 2020 to april 2022. we focused on the semantic shift phenomenon of vaccine-and symptom-related entity words and observed their semantic shift patterns at different stages of the pandemic, as well as their potential correlations with real epidemic trends. the main contributions are summarized as follows: 1) we propose a dynamic word embedding method that can adaptively detect semantic shift phenomena in longitudinal corpora without manual annotation; 2) we integrate it with named entity recognition (ner) and large language model (llm)-based entity normalization to systematically investigate the semantic shift in extensive and longitudinal social media corpora during the covid-19 pandemic; 3) our experimental results reveal the semantic evolution patterns of vaccineand symptom-related topics during the covid-19 pandemic, offering a new perspective on the study of epidemic evolution and its social impact. this provides a novel paradigm for leveraging computational linguistics in social and medical research. ## related work semantic shift analysis is a crucial research problem in the fields of computational linguistics and natural language processing. over time, the semantics of words evolve, reflecting changes in language usage and socio-cultural dynamics. researchers have recently proposed various data-driven methods to automatically detect semantic shift phenomena at the word level. early studies on semantic shift primarily relied on word frequency statistics, analyzing the temporal variation of word frequencies in books @xcite and in web search queries @xcite . however, it has been noted that word frequency changes are not always directly correlated with semantic shifts @xcite , with frequency information alone being insufficient to accurately capture subtle semantic variations. distributional semantics has been proven to be an effective tool for semantic shift analysis @xcite . by leveraging word co-occurrence statistics, distributional representations of words can be learned, directly encoding their semantics as vectors. @xcite pioneered quantitative analysis of semantic shifts with word vector updates to dynamically model semantic evolution, with subsequent studies employing various word embedding algorithms, such as latent semantic analysis by @xcite and local mutual information by gulordava and @xcite . @xcite introduced predictive word embedding models (e.g., word2vec) to the semantic shift detection task, followed by works comparing performance of different types of word embedding models @xcite which found that predictive models like skip-gram with negative sampling (sgns) outperform explicit counting methods. other studies @xcite have looked at characterizing semantic shift types, such as the emergence of novel senses, splitting and merging of word senses, which falls under fine-grained semantic shift analysis. hamilton et al. (2016a) explored the distinction between cultural and linguistic shifts as the two modes of semantic evolution, with the former closely related to socio-cultural changes and manifesting as global displacement in the word vector space, and the latter reflecting the internal evolution of language and characterized by changes in local neighborhoods. semantic shift analysis through the topic evolution lens is explored through dynamic topic models to track the temporal variation of topic distributions @xcite or temporal evolution of google search topics @xcite . the emergence of contextualized word embeddings capable of capturing diversity and subtle changes in word semantics in pre-trained language models such as elmo @xcite and bert @xcite has led to the use of bert for semantic shift detection, with a novel representation splitting method to improve detection accuracy @xcite . subsequent studies, such as those by @xcite have con-preliminary embedding slice merged embedding slice gradual updating considering temporal continuity word co-occurrence preliminary embedding (3) dynamic word embeddings update (2) adaptive selection of time slices (1) co-occurrence analysis dynamic updating considering semantic information of the current and previous time slice (4) semantic shift detection and analysis semantic similarity computation firmed their superior performance over static ones with systematic evaluations of different types of contextualized word embeddings on this task. moreover, some studies focused on the interpretability of semantic shift detection. @xcite proposed an interpretable semantic shift model based on attention mechanisms, explaining the reasons for word semantic evolution by visualizing attention weights. @xcite employed causal inference methods to investigate the sociocultural driving factors behind lexical semantic shifts. these works contribute to a deeper understanding of semantic shift phenomena from both computational and humanistic perspectives. in comparison to the above-mentioned methods, suppressing high-frequency core words and dynamically updating weights ensure that we can detect stable semantic shifts in embeddings. regarding the analysis of covid-19 discussions on social media, several studies have explored this topic from different perspectives. @xcite investigated the role of machine learning in identifying misinformation during the pandemic, and explored how older adults interacted with online information, often finding it difficult to verify and preferring traditional media sources. @xcite focused on the metaphorical language used in social media, particularly war-related terms, showing how certain metaphors shaped public conversations around the virus and its effects. @xcite analyzed the relationship between pandemic trends, such as infection rates, and shifts in public sentiment on twitter, using causal modeling to distinguish between events that influenced public attention and those that were merely correlated. these studies collectively offer insights into the dynamic nature of covid-19 discourse online, shaped by both statistical methods and human perception. ## methodology this section introduces the proposed unsupervised dynamic word embedding method for detecting and analyzing semantic shifts of words in detail. the overall framework is shown in figure 1 . our method leverages word co-occurrence statistics and adopts adaptive selection and time slice dynamic updating strategies to analyze and capture the evolution of word semantics over time. ## conclusion and future work this study proposes an innovative unsupervised dynamic word embedding method to capture the longitudinal evolution of word semantics in social media texts. we apply this method to symptom and vaccine semantic analysis using large-scale covid-19 twitter data, achieving promising results. our method can effectively identify the dy- (b) jun 2021 -apr 2022 (a) feb 2020 -sep 2020 namic patterns of symptoms and vaccine semantics at different stages of the epidemic. by introduc-ing temporal dynamics and iterative update strategies, this method overcomes the limitations of tra-ditional static word embeddings, achieving longitudinal tracking of semantic content in social media texts. additionally, we analyze the semantic interactions between different symptoms and between symptoms and vaccines, revealing their synergistic effects over time. in the future, we plan to expand the application of this method in the field of vertical semantic analysis. on one hand, we will explore more advanced time series modeling techniques to more accurately capture the evolving patterns of word semantics over time. on the other hand, we will integrate semantic shift analysis with technologies such as social network analysis and sentiment analysis to investigate the rich semantic relationships and patterns embedded in social media texts, such as identifying precursor signals of sudden events and revealing the driving factors behind the evolution of public opinion.