Understanding Weekly COVID -19 Concerns through Dynamic Content-Specific LDA Topic Modeling (2020)

Abstract: The novelty and global scale of the COVID-19 pandemic has lead to rapid societal changes in a short span of time. As government policy and health measures shift, public perceptions and concerns also change, an evolution documented within discourse on social media. We propose a dynamic content-specific LDA topic modeling technique that can help to identify different domains of COVID-specific discourse that can be used to track societal shifts in concerns or views. Our experiments show that these model-derived topics are more coherent than standard LDA topics, and also provide new features that are more helpful in prediction of COVID-19 related outcomes including social mobility and unemployment rate.

## introduction in early 2020, the entire world slowly became aware of a severe respiratory disease @xcite with often catastrophic consequences @xcite . since then, covid-19 has infected millions of people worldwide and is on a trajectory to cause more than 1 million deaths globally before the end of 2020 @xcite . government and public health agencies have sought to mobilized and promote protective measures @xcite as scientific efforts rapidly build a foundation of knowledge to treat the disease, and economies have experienced rapid drops in employment @xcite . these ongoing changes make the covid-19 pandemic a time of immense and multifaceted social change that is rapidly evolving and largely, if imperfectly, narrated from millions of voices on social media. user generated discourse like social media provide rich and continuous data on the evolution of significant world events -be it pandemics, hurricane relief, earthquakes, or wildfires @xcite . given the rapidly evolving nature of such public health emergencies, the ability to extract and quantify the progression of topics can provide a unique window into the concurrent societal change. realizing this objective is partially met through the goals of topic modeling, such as latent dirichlet allocation (lda; @xcite ). however, tracking significant world events present 2 challenges to standard topic models: (1) the need for rapidly evolving topics rather than topics from a single snapshot of language at a time, and (2) the need to focus on event-relevant lexical patterns rather than general patterns. latent dirichlet allocation (lda) is one of the most commonly used topic modeling methods, whereby probabilities of words belonging to topics (clusters of semantically related words) are derived from textual data. not only can this provide a good set of features for predictive models @xcite , but also a tool for generating hypotheses and gaining insight in a manner easily interpretable by humans @xcite . in this paper, we present and evaluate modifications to lda, addressing the two aforementioned challenges, to focus on capturing topics that can characterize evolving interests specifically for covid-19. addressing such challenges enables several applications including monitoring the impact of a specific event on social, emotional, mental well-being and behaviours @xcite . building on previous work in online topic modeling @xcite , we propose the creation of short-interval dynamic topics that are updated on a regular basis (weekly or monthly). furthermore, because words in social media posts cover a wide variety of domains (even when limited to posts containing covid-19 keywords), we introduce a content-specific preprocessing step that focuses the lexicon on the domain. this limits the generation of incoherent or general domain topics -rather than a single general health care topic, multiple health care topics may emerge, such as those related to vaccines and testing. this is especially important in the context of rapidly evolving public health emergencies like covid-19, where conversations around physical distancing and protective measures have evolved as national policy has changed @xcite . our contributions include the formalization and evaluation of methods for topic modeling over (1) time series language data and (2) content focused lexical patterns; ## data we built a covid-19 corpus from publicly available twitter data (march 12, 2020 to the end of june 2020; 2.6 million tweets), pulled from the streaming api using a set of covid-19 keywords. we also took a sample of tweets pulled from the random 1% stream, which did not contain the covid-19 keywords, as our matched tweets. we break this corpus into two sets of weekly and monthly data sets. the former, which is used for the coherence and real time tracking experiments, spans 5 weeks of tweets and contains 150,000 tweets per week for each of the covid-19 and matched tweets. the latter, used for prediction experiment, spans 4 months and contains 500,000 tweets per month. for the prediction task, we collect two monthly us county covid-19 related outcomes: we use the safegraph places patterns foot_1 data, which contains aggregated and anonymized foot traffic data for 6 million points across the us. we use three months of mobility data from april to june, 2020. we also collect two months (april and may, 2020) of yearly-adjusted unemployment rates as reported by the us bureau of labor statistics (bls) foot_2 . ## experiments we perform three tasks to evaluate our proposed method. first, we consider weekly topic coherence as an intrinsic evaluation metric. next, we manually compare the evolution of hand selected topics to real-world events for real time tracking. finally, as an extrinsic evaluation metric, we use our topics as feature for predicting monthly mobility and unemployment. for each of our three experiments, we create 40 weekly and monthly topics (for each lda method) using the java-based mallet software package @xcite , which implements gibbs sampling @xcite . for each lda method, parameters are kept constant (α = 5/n , β = 0.1), where n is the number of desired topics. additionally, stop words are removed and the number of unique tokens is constant at 8,000. all text pre-processing as well as the prediction task is done with the python package dlatk @xcite . coherence score can be obtained as the summation of coherence score of each pair of words from the top selected words of that topic. for the coherence of pair of words there are various options, where we use point-wise mutual information (pmi) as suggested by @xcite : our baseline is a single iteration of gibbs sampling for standard lda and is used for normalizing all scores. figure 1 shows the results of our coherence task. not surprisingly, all four lda methods outperform the single iteration baseline. we see a small increase in coherence when using the dynamic approach over standard lda. significant increases (above both the standard and dynamic approaches) occur when using the cs method, as determined by a t-test (@xmath0.05). finally, we see marginal increases in coherence (above the cs method) when using the the dynamic cs approach. real time tracking for the tracking task, the goal is to see if our topics are evolving alongside real world events. as such, we hand select three topics, labeled as treatment, economy and demand and show their evolution over a 5 weeks period. the timeline in figure 2 shows the top eight weighted words within each topic, in addition to plotting notable covid-19 events. the demand topic starts with mentions of toilet paper and panic buying and moving to masks and general supplies. the treatment topic mentions hydroxychloroquine around the time the fda approved emergency use (march 29) and when trump was frequently mentioning the drug (april 3 through 5). finally, the economy topic starts with calls for emergency federal help ("sign", "act", "pass") and then moves onto more specific information around the stimulus bill ("2 trillion" and "congress") before stabilizing. prediction using a 10-fold cross validation setup, we predict two monthly covid-19 related outcomes: social mobility and unemployment rates. we extract monthly topic representations (standard, dynamic, cs and dynamic cs) for us counties who have written at least 7,500 words per month (i.e., each observation is a us county and is represented as a bag-of-topics). here, we use language features for a given month to predict the outcomes for the following month (e.g., topics from march are used to predict outcomes in april). for each month, we perform a 10-fold cross-validation of a regularized ridge regression and report the pearson r between our predicted values and the actual rates. we use unigram features as the baseline and compared our gain by adding standard lda features vs. dynamic cs lda features. results in ## discussion discussions on social networks are rapidly changing, especially those centered around covid-19. as such, the goal of the current study was to see if we could reasonably monitor those changes over time using a novel lda topic-modeling approach, namely dynamic content-specific lda. results showed that we could reliably track topic evolution and leverage those dynamics to predict changes in real work outcomes (mobility and unemployment) we found that many such observations could be readily tracked, suggesting that we could reliably track discussions, even when the specific words or concerns used to reference that topic change. this has a direct application to covid-19 as it may help determine appropriate shifts in public health strategy as topics evolve towards consensus. additionally, insofar as topics involve and reliably track symptoms, then evolution in reported symptoms may help in efforts to map out differences in strains or in disease evolution. it is also worthwhile to note that this model can be used to track any evolving discussion over time. indeed, many discussions both in the field of public health or medicine and in other fields, such as understanding financial shifts or changes to public acceptance of known falsehoods or conspiracies, are discussed regularly online and may be tracked as they evolve. the evolution of such discussions can be telling, we think, and the mapping of that evolution warrants further interrogation. a central goal in epidemiological modeling is monitoring psychiatric and health outcomes, and their correlates, as they shift over time. this study helps us to better identify and track the discourse around covid-19 via an online social network and also helped to characterize changes in topics as they evolved over time. as such, lda topic analysis identified and helped to characterize changing interests in both covid-19 and its societal effects as they emerged over time. this study suggests that further research should seek to understand how these topics are associated with the distribution of covid-19 cases and changes in mental health.