Detecting Emerging Symptoms of COVID -19 using Context-based T witter Embeddings (2020)

Abstract: In this paper, we present an iterative graph-based approach for the detection of symptoms of COVID-19, the pathology of which seems to be evolving. More generally, the method can be applied to finding context-specific words and texts (e.g. symptom mentions) in large imbalanced corpora (e.g. all tweets mentioning #COVID-19). Given the novelty of COVID-19, we also test if the proposed approach generalizes to the problem of detecting Adverse Drug Reaction (ADR). We find that the approach applied to Twitter data can detect symptom mentions substantially before to their being reported by the Centers for Disease Control (CDC).

## introduction the covid-19 pandemic has interrupted many everyday behaviors. sars-cov-2 is a relatively new virus and gaps in knowledge persist about how it affects the body, and consequently, its symptoms and symptom severity. in the early phases of the pandemic, patients and providers in affected areas used social media to exchange information about symptoms and clinical treatment @xcite . during covid-19, the use of social media has increased dramatically (>20%) as individuals shelter in place @xcite . while social media is non-representative and contains misinformation @xcite , it provides an open forum for individuals to share their perceptions, concerns @xcite , understanding of health and science @xcite , and mental health status during emergencies when wide spread polling may not be available @xcite . social media could enable early symptom discovery for diseases such as covid-19 where the pathology is not completely known and our knowledge of it is evolving @xcite . the most prominent symptoms such as fever, cough, and shortness of breath were known early on during the covid-19 pandemic. however, others such as changes in smell/taste, body aches, and diarrhea were added later to the symptom list by the cdc @xcite . using social media to gather information on public health is a growing focus of research, with a special emphasis on discovering side effects of drugs (pharmacovigilance) @xcite , often using labeled datasets to build supervised machine learning models @xcite . we propose a graph-based natural language processing framework to automatically detect emerging symptoms using twitter data. our approach is built on the hypothesis that by identifying token embeddings that capture the context of symptom mentions, new tokens used in a similar context can be identified through embedding similarity @xcite . our approach shares similarities with the idea of lexicon development @xcite , which uses an unsupervised graph-based approach for labeling new words given a few labeled words. however, the graph is initiated with words of interest that have already been identified. our method's focus on a specific context allows it to search through large imbalanced corpora to identify context-specific (e.g. symptoms) tweets. this differentiates it from previous works by @xcite ) that identify domain specific lexicon. further, the approach by @xcite relies on a domain specific corpus and topic modeling to build a lexicon, which would require the construction of a symptom-specific covid-19 corpus. ## method as is the case with several applications involving creating word lists associated with a construct or topic @xcite , symptom mentions associated with covid-19 come in different forms and shapes -often difficult to curate in advance @xcite . the approach we propose assumes that we know at least one word of interest (i.e., a seed word) along with few corresponding seed texts where the seed word has been used in the desired context. for the case of emerging symptom detection, cough, a seed text could be 'i have a dry cough, chest pain and feeling lethargic as hell plus a headache'. ## discussion the covid-19 pandemic evolved in a global climate of confusion and uncertainty. the professional and lay public alike speculated on disease course, severity, and symptoms. the covid-19 symptoms first observed appeared to be largely nonspecific to covid-19 (e.g., cough, fever). finding covid-specific symptoms (such as the sudden loss of the sense of smell) is important and potentially of clinical significance as large populations are being risk-assessed @xcite . the "digital exhaust" of social media encodes in-formal case reports of symptoms and discussions of media content about the virus alike. in principle, it could allow for the generation of a "master list" of covid-19 symptom candidates, which the public health and medical community can, in turn, consider for further evaluation as covid-specific markers @xcite . in this study, we present an iterative learning approach to generate such a "master" list of covid-19 symptoms, using the identification of words matching a specific symptom context. our approach is intended to work at the intersection of human-machine interaction and provide a ranked list of most important keywords for a user to explore and validate. a typical workflow would involve users from the medical/ pharmacovigilance domain inputting a small number of known keywords of interest, detecting the emerging signals (symptoms/adr) output by the model, and validating them using the corresponding text. this allows for very early detection of symptoms/adr from social media platforms without the need to manually scavenge large corpora. our initial experiments with word2vec @xcite , glove @xcite , and latent dirichlet allocation @xcite for the same task, did not perform well since the similarity between tokens was the same irrespective of the context in which the word was being used. so with 'cough' as the seed word, highly correlated words like 'fever' were detected by these approaches, while words like 'congestion', 'nausea', 'diarrhea' were not. consequently, we developed an iterative graph-based training approach using bert embeddings that are able to take into account the context in which words are used. through a preliminary evaluation, our approach shows high sensitivity in detected words. the approach detected headache, chills, sore throat, diarrhea, and other symptoms around a month before the cdc reported them (figure 1 ). given the novelty of covid-19, the current method is hard to evaluate. recall and other metrics that depend on false negatives are not useful as there is no definitive 'ground truth'. this is even more true for covid-19 as the symptoms are evolving and newer symptoms are being observed even after several months from the first case. we, therefore, considered the approach in the more studied context of detecting adverse drug reactions and show that the approach generalizes to this domain. the annotations in adr vary from very specific entities like 'pain', 'headache' to very broad phrases like 'a tingling sensation' @xcite . though our approach does not extract multi-word terms, given that the typical workflow is expected to focus on the validation of symptoms using their corresponding full-length texts, the utility of multiword terms is minimal. the approach relies on the context embedding contribution parameter (k). by varying k, we observe a phenomenon analogous to 'exploration vs exploitation' @xcite , which, in principle, means that this method can be calibrated for different use cases. in the early phase of the disease, for example, a low k parameter may be chosen to aid in the generation of symptom candidates to be considered in light of the emerging clinical literature on covid-19 and known physiological and biological interactions in the human body. a high k may be chosen to yield the subset of covid-19 symptoms that are more robustly associated with the disease, at the cost of missing infrequent (albeit potentially specific) disease markers. in summary, this study demonstrates that the digital traces of social media can be mined effectively to detect emerging symptoms in the population during a public health emergency.