An Exploratory Study on Temporally Evolving Discussion around Covid-19 using Diachronic Word Embeddings (2021)

Abstract: Covid 19 has seen the world go into a lock down and unconventional social situations throughout. During this time, the world saw a surge in information sharing around the pandemic and the topics shared in the time were diverse. People’s sentiments have changed during this period. Given the wide spread usage of Online Social Networks (OSN) and support groups, the user sentiment is well reflected in online discussions. In this work, we aim to show the topics under discussion, evolution of discussions, change in user sentiment during the pandemic. Alongside which, we also demonstrate the possibility of exploratory analysis to find pressing topics, change in perception towards the topics and ways to use the knowledge extracted from online discussions. For our work we employ Diachronic Word embeddings which capture the change in word usage over time. With the help of analysis from temporal word usages, we show the change in people’s option on covid-19 from being a conspiracy, to the post-covid topics that surround vaccination.

## introduction analysing and estimating the impact of changes in social lives of people from time to time is key in digital humanities. with the advent of online social networks (osns) it has become easy to access what people have been sharing their lives online, good or bad and seeking support in the online communities. during the pandemic, people seek help and they have also shared information with one another. to understand how people's lives were affected, and to contribute towards digital humanities, we look at conversations on covid-19 during the pandemic. we listed the wide variety of topics discussed, and the support from each other when they were stuck with the illness. covid-19 pandemic has been a huge disaster for humanity. it has changed the way we live forever during the ensued lockdown from the pandemic. we saw a lot of community support and huge participation in supporting each other during the testing times. similarly, covid-19 is the first known global pandemic in the internet age, when the information disposal is rapid and the ability to communicate anywhere on the earth takes seconds. it is of immense significance to see how people communicated around coordinating and how people supported each other during the times, as we can assess the ability of the world wide web in effect. the information revolution in the form of worldwide web has also come with its own challenges like fake news, conspiracy theories, hate speech, etc. it is of particular interest to understand how the conspiracies and the narrative around covid-19 has changed ever since the pandemic broke. for example, at the beginning of the pandemic, there has been a lot of speculation that covid-19 is a hoax, and the virus is just political propaganda. such narratives can be found around the world, including the united states, india, china, and japan. however, it did not take long for people to realize that covid-19 is, in fact, a pandemic. so, during the same period, people started discussing online about the pandemic and the narrative around conspiracy theories. in this work, we aim to understand how the topics involved in covid-19 have changed since the inception of the pandemic till date. for this work, we use a data set taken from reddit, which is a popular social network built around smaller communities called subreddits. particularly, we choose the subreddit r/covid19positive 1 . our data set contains all posts and comments made since the inception of the subreddit, and we use the data to understand the narratives around covid-19 on how people's perception has changed. specifically, we aim to assess the change in word usages on the basis of proximity among different words in the initial stages of the pandemic till date. the proximity of words and word usage is of importance because, togetherness of words gives us an idea on how people are thinking and what people are talking about. so, to estimate the word usage and proximity, we use robust machine learning methods such as diachronic word embeddings @xcite . the diachronic word embeddings @xcite take different embedding models trained on text corpora which are linked together in time. the text corpora are expected to evolve over time, and the temporal usage of words is captured by taking smaller snapshots of the corpus. to employ the technique, we train individual models and then align the models in such a way that the same words stay in relatively similar positions across time; thereby making the change in word embeddings relative. by closely looking at snapshots and the drift of words among snapshots, we can estimate the change in word usage. multiple works have studied and seen a change in word usage throughout human history with the lens of corpora. covid-19 being two years old, we aim to do a word evolution estimation within these two years for our work. we split our corpus into month-long sub-corpora and train the embedding models on these individual corpora. later, we align the corpora to maintain the word similarity across time. once we align the word similarity across time, we estimate the divergence of words or drift from each other. the divergence and drift give us information on how the words have changed. some interesting observations from our work include the usage of 'conspiracy'; at the beginning of the pandemic, the word and covid-19 are closely related, and as time progresses, the word conspiracy disappears from the neighborhood of covid. similarly, covid-19 is known for its symptoms such as cough and cold. during the initial phase, these words are seen away from 'covid' or 'virus'. as time progresses and humanity goes into waves of huge infections, the words come together. humanity was lucky enough to produce vaccines in a short time, a narrative around vaccines and their evolution is also interesting to look at. although a lot of vaccine-related discussions were taking place in the initial days of covid, the usage of vaccines along with infection has increased recently (as of @xcite . with our work, we also aim at developing a system that can be used by people with little to no technical knowledge of how word embeddings operate and obtain the information we extract. in summary, our work focuses on : in this section, we discuss the works that have studied social media websites during the covid-19 crisis. in an early work, @xcite the authors have collected chinese articles about covid-19 and reported the user sentiment around the pandemic. they have used news articles as the dataset. similar works are seen in brazil (de melo and figueiredo, 2021), that used twitter and four european nations @xcite based on news articles. we see a common pattern in the works which is to collect a publicly available dataset, use a language model and report the findings along side the real-world happenings. another widely observed theme in literature is to extract medical information from publicly available articles. the works @xcite @xcite attempt at extracting medical information such as symptoms, post-covid medical status, topics around vaccination using osn data. the osn of choice in the majority of these works is reddit. we place our work at an intersection of the above-mentioned literature. we note the absence of literature on the evolution of themes, scientifically extracting and evaluating the change in discussions. our work bridges the gap in the literature. ## methodology for our work, to study the evolution in perception towards the pandemic related to covid-19, we choose the subreddit r/covid19positive as our data source. reddit is a community-centric social network where users form communities specific to topics and intents called subreddits. the r/covid19positive subreddit is "a place for people who came back positive for covid19 can share your stories, experiences, answer questions and vent!" -according to the subreddit description. as the subreddit contains discussions related to support, experience, and qna by people tested positive, our data will be rich in capturing diverse contexts while being very specific to our topic of interest. with more than 124k users, the subreddit contains discussions ranging from users seeking support from the loss of a family person, to users discussing the geopolitical issues surrounding the pandemic. next, we discuss the dataset preparation. ## results and discussion having established the methodology and visualizations that aid us in understanding word evolution, we now discuss the results. ## concluding remarks in this work, we have used diachronic word embeddings to assess the temporal evolution of topics around covid-19 pandemic. for one and half years, we take the help of r/covid19positive subreddit. in order to conduct our study, we have captured the users throughout the three waves of the united states. we have closely followed the pandemic in the united states because our conversations have followed the peaks from the united states. in our work, we split the corpus into three, and compared the three sub-corpora; each subcorpora capturing a different phases of the pandemic. in the first wave, we've had a lot of uncertainty around covid, and a lot of conspiracy theories were floating around. we find that by the second wave, people got serious, and there was a lot of support going on in the community. people were discussing vaccines in the latter half of the second wave. however, during the third wave, we have seen that people were discussing being vacci-nated and getting a positive result on their covid test after vaccination. we also demonstrate an important aspect of post-covid symptoms, which are related to long-standing illness such as cancer and heart failure. with this work, we demonstrate the utility of diachronic embeddings in the real world. we also demonstrated the application of diachronic embeddings on a very short time period unlike any previous literature or research. in the future we aim to build a dashboard that can be easily used by domain experts, medical professionals and journalists to access the information we extracted in our work.