Characterizing drug mentions in COVID -19 T witter Chatter (2020)

Abstract: Since the classification of COVID-19 as a global pandemic, there have been many attempts to treat and contain the virus. Although there is no specific antiviral treatment recommended for COVID-19, there are several drugs that can potentially help with symptoms. In this work, we mined a large twitter dataset of 424 million tweets of COVID-19 chatter to identify discourse around drug mentions. While seemingly a straightforward task, due to the informal nature of language use in Twitter, we demonstrate the need of machine learning alongside traditional automated methods to aid in this task. By applying these complementary methods, we are able to recover almost 15% additional data, making misspelling handling a needed task as a pre-processing step when dealing with social media data.

## introduction the world health organization (who) defines the coronavirus disease (covid-19) as an infectious disease caused by a newly discovered coronavirus and declared it a pandemic on march 11, 2020 (world health organization). as of june 26, 2020, 9,764,997 cases were confirmed worldwide with 492,807 deaths and 4,917,328 cases recovered. social media platforms like twitter and reddit contain an abundance of text data that can be utilized for research. over the last decade, twitter has proven to be a valuable resource during disasters for many-to-many crisis communication @xcite @xcite . recently, several works @xcite @xcite have provided insights on the treatment options and drug usages for covid-19. we utilized the largest available covid-19 dataset @xcite curated using a social media mining toolkit (smmt) @xcite . version 15 of the covid-19 dataset was utilized for our experiments since it was the latest released version at the time of experiments. this dataset consists of tweets related to covid-19 from january 1, 2020 to june 20, 2020. we automatically tagged ~424 million tweets using a drug dictionary compiled from rxnorm (national library of @xcite with 19,643 terms and validated in @xcite and @xcite . ## methods in order to identify drug-specific tweets related to covid-19, we applied the drug dictionary on the clean version of the dataset. with the presence of retweets, any signal found would be greatly amplified and hence only unique tweets must be utilized to identify the signals. the cleaned version of the dataset consists of 104,512,658 unique tweets with no retweets. we are only using the english language tweets, which are 67% (~70 million tweets) of them. the spacy annotator utility from the social media mining toolkit @xcite was utilized to tag the clean tweets. a total of 723,129 tweets were identified as containing one or more drug terms. however, complex medical terms, such as medication and disease names, are often misspelled by users on twitter @xcite . using only the keywords with correct spellings leads to a loss of potentially important data, particularly for terms with difficult spellings. some misspellings also occur more frequently than others, implying that they are more important for data collection and concept detection. to identify the misspelled drug tweets, we initially identified the top 200 drug terms from the 723,129 drug tweets tagged. additionally, early in the stages of this work, a preliminary report from a large study of 1,063 patients showed that hospitalized patients who were administered remdesivir recovered faster than those who got a placebo @xcite . however, the version of rxnorm we have utilized for the original drug dictionary did not have remdesivir since it was an investigational drug and was later added to rxnorm in april 2020 (bulletin, 2020). we manually added this drug term for our analysis. in this work, we demonstrate the tradeoff between generating misspellings, using generated word-embedding misspellings, and auto-correcting the spellings at the text level. for this purpose, we employed four different methodologies to acquire additional data. the first methodology involves a machine learning approach called qmisspell @xcite . the redmed (lavertu and altman, 2019) model's word embeddings were utilized in qmisspell to generate misspellings. this redmed model has an embedding size of 64 dimensions with a window size of 7 and created utilizing the health-oriented subset of reddit. qmisspell relies on a dense vector model learned from large, unlabeled text, which identifies semantically close terms to the original keyword, followed by the filtering of terms that are lexically dissimilar beyond a given threshold. the process is recursive and converges when no new terms similar (lexically and semantically) to the original keyword are found. a total of 2,056 unique misspelled terms were generated using qmisspell. we examined all the misspellings and eliminated several terms (example: heroes, heroine generated from the original keyword heroin) which are common terms in the english vocabulary and are not misspellings of drug terms. post elimination, a total of 1,932 terms are identified as the misspelt terms. the second methodology utilizes keyboard layout distance for generating the misspellings. for each term, each letter is replaced with the closest letter on the qwerty keyboard. this process is recursive and ceases when it has looped through every letter in the term. for example, the term cocaine has 68 misspelled terms which vary from xocaine to cocaint. after eliminating the common vocabulary and duplicates, a total of 15,754 terms are identified as misspelled terms for this methodology. we text tagged the clean dataset utilizing the misspelled terms from the methodologies. the third methodology employs a spelling correction module called symspell @xcite which corrects the spelling errors at text level before text tagging. this method is built using the symmetric delete spelling correction algorithm which reduces the complexity of edit candidate generation and dictionary lookup for a given damerau-levenshtein distance @xcite . this method generates terms with an edit distance (deletes only) from each dictionary term and adds them together with the original term to the dictionary and searches the dictionary. this has to be done only once during a pre-calculation step. for a word of length n, an alphabet size of a, an edit distance of 1, there will be just n deletions, for a total of n-terms at search time. the symmetric delete spelling correction algorithm (symspell) reduces the complexity of edit candidate generation and dictionary lookup by using deletes only instead of deletes + transposes + replaces + inserts @xcite . this methodology requires a frequency dictionary of the common terms. since we are extracting only english drug related tweets, we used the default english frequency dictionary provided by symspell and added a drug frequency dictionary generated using ~60 million clean tweets. additionally, we manually included the top covid-19 keywords @xcite with frequencies in our initial tagging, since these terms are found in large numbers. the symspell algorithm corrects each term in a tweet text to the closest matching term in the dictionary. the corrected tweet is then sent to the smmt spacy utility for tagging. a total of 17,686 misspelled drug terms were generated for the top 200 drug terms using the qmispell and keyboard layout methods. the fourth methodology utilizes textblob (textblob), a python library for processing textual data. it provides a consistent api for diving into common natural language processing (nlp) tasks such as part-of-speech tagging, noun phrase extraction, spelling correction and more. the spelling correction feature of the textblob employs peter norvig's spelling correction algorithm @xcite which generates a candidate model by a simple deletion to a word (remove one letter), a transposition (swap two adjacent letters), a replacement (change one letter to another) or an insertion (add a letter). this method was applied on each tweet text before tagging. unfortunately, this implementation is highly inefficient, taking around 60 hours of execution time for every 600,000 tweets. we were only able to run this on 20 days of data, and only present results in the computational evaluation table. all experiments in this work are presented in the following section and focus on the first three methods outlined. ## results method total number of terms in dictionary total tagged term in dataset spacy + rxnorm dictionary 19,643 1,483,691 spacy + qmisspell dictionary (only) 1,932 192,619 spacy + keyboard layout generated misspellings dictionary (only) 15,754 135,981 spacy on symspell corrected text + rxnorm dictionary 19,643 1,632,951 ** unsurprisingly, hydroxychloroquine was the most tweeted drug found on the dataset with chloroquine coming in as close second, as shown in the first item to note is that for both the keyboard and qmisspell methods, these times need to be added to the baseline times as they are executed independently in addition to the base text tagging. however, they are not even close in terms of computational expense to the symspell or textblob spell correction, which are quite expensive. for textblob, we only processed 20 days of data before aborting as this package took on average 60 hours of execution time per 600,000 tweets. in order to show the overlaps of terms tagged with each of the different methods explored, figure 2 shows the total overlap in percent between the new terms tagged. there is a 35% overlap between the 3 methods since there are many generated terms that are produced by all methods, these are usually the more frequent ones or typical ones. showing that a brute force approach (keyboard based method) might not be ideal to generate misspellings on large sets of terms as it would add considerable computational expense. finally, in ## conclusions in this work, we focused on extracting discourse related to the potential drug treatments available for covid-19 patients. while seemingly a trivial task, we show that enough consideration is needed for the proper way to deal with the constant misspellings found in twitter data. we have shown that with a combination of methods we can identify around 15% additional terms, which would have been lost. with data being quite limited and not easily available, it is important to apply the proper techniques to identify the largest subset of data possible. we provide a quantifiable evaluation by generating misspellings utilizing two different methods and automated spell check using symspell. while the keyboard layout method is a fully automated method to generate misspellings, the qmisspell method generates misspellings based on language models. we have shown that this work is indeed needed to be performed in order to make a thorough evaluation of the discourse regarding potential drug treatments for covid-19 patients.