Navigating the Kaleidoscope of COVID -19 Misinformation Using Deep Learning (2021)

Abstract: Irrespective of the success of the deep learning-based mixed-domain transfer learning approach for solving various Natural Language Processing tasks, it does not lend a generalizable solution for detecting misinformation from COVID-19 social media data. Due to the inherent complexity of this type of data, caused by its dynamic (context evolves rapidly), nuanced (misinformation types are often ambiguous), and diverse (skewed, fine-grained, and overlapping categories) nature, it is imperative for an effective model to capture both the local and global context of the target domain. By conducting a systematic investigation, we show that: (i) the deep Transformer-based pre-trained models, utilized via the mixed-domain transfer learning, are only good at capturing the local context, thus exhibits poor generalization, and (ii) a combination of shallow network-based domain-specific models and convolutional neural networks can efficiently extract local as well as global context directly from the target data in a hierarchical fashion, enabling it to offer a more generalizable solution.

## introduction since the start of the coronavirus or covid-19 pandemic, online social media (e.g., twitter) has become a conduit for rapid propagation of misinformation @xcite . although misinformation is considered to be created without the intention of causing harm @xcite , it can wreak havoc on society @xcite @xcite and disrupt democratic institutions @xcite . misinformation in general, and covid-19 misinformation in particular, has become a grave concern for the policymakers due to its fast propagation via online social media. a recent study shows that the majority of the covid-19 social media data is rife with misinformation @xcite . the first step towards preventing misinformation is to detect misinformation in a timely fashion. building automated systems for misinformation detection from social media data is a natural language processing (nlp) task. various deep learning models have been successfully employed for this type of nlp task of text classification @xcite @xcite @xcite . these models learn language representations from a domain, which are then used as numeric features in supervised classification. due to the prohibitive cost of acquiring labeled data on covid-19 misinformation, training deep learning models directly using the target data is not a suitable approach. background. an alternative approach for detecting covid-19 misinformation from small labeled data is transfer learning @xcite . the dominant paradigm of transfer learning employs a mixed-domain strategy in which representations learned from a general domain (source data) by using domain-agnostic models are transferred into a specific domain (target data) @xcite . specifically, it involves creating a pre-trained model (ptm) that learns embedded representations from general-purpose unlabeled data, then adapting the model for a downstream task using the labeled target data @xcite . two types of neural networks can be used to create ptms, i.e., shallow and deep. the shallow models such as word2vec @xcite and glove @xcite learn word embeddings that capture semantic, syntactic, and some global relationships @xcite of the words from the source text using their co-occurrence information. however, these ptms do not capture the context of the text @xcite . on the other hand, deep ptms can learn contextual embed-dings, i.e., language models @xcite . two main approaches for creating deep ptms are based on sequential and non-sequential models. the sequential recurrent neural network @xcite based model such as elmo (embeddings from language models) @xcite is equipped with long short-term memory to capture the local context of a word in sequential order. the non-sequential transformer @xcite based models such as openai gpt (generative pre-training) @xcite , bert (bidirectional encoder representation from transformer) @xcite , and xlnet @xcite utilize the attention mechanism @xcite for learning universal language representation from general-purpose very large text corpora such as wikipedia and book-corpus @xcite as well as from web crawls @xcite . while gpt is an autoregressive model that learns embeddings by predicting words based on previous predictions, bert utilizes the autoencoding technique based on bidirectional context modeling @xcite . xlnet leverages the strengths of autoregressive and autoencoding plms @xcite . unlike the transformer-based deep ptms, the shallow word2vec and glove as well as the deep elmo ptms are used only as feature extractors. these features are fed into another model for the downstream task of classification, which needs to be trained from scratch using the target data. the deep ptm based mixed-domain transfer learning has achieved state-of-the-art (sota) performance in many nlp tasks including text classification @xcite . irrespective of the success of the mixed-domain sota transfer learning approach for text classification, there has been no study to understand how effective this approach is for navigating through the kaleidoscope of covid-19 misinformation. unlike the curated static datasets on which this approach is tested @xcite , the dynamic landscape of the covid-19 social media data has not been fully explored. some key properties of the covid-19 data hitherto identified are: (i) the covid-19 misinformation spreads faster on social media than any other form of health misinformation @xcite . as a consequence, the misinformation narrative evolves rapidly @xcite . (ii) the covid-19 misinforma-tion categories are heavily-skewed @xcite and fine-grained (memon and carley, 2020). (iii) the covid-19 social media misinformation types are often ambiguous (e.g., fabricated, reconfigured, satire, parody) @xcite and categories may not be mutually exclusive (memon and carley, 2020). these properties pose a unique challenge for the mixed-domain sota transfer learning approach for creating an effective solution to the covid-19 misinformation detection problem. previously, it has been shown that the transfer learning approach generalizes poorly when the domain of the source dataset is significantly different from that of the target dataset @xcite . on the other hand, domain-specific models (dsm), which learn representations from domains that are similar to the target domain, provide a generalizable solution for the downstream nlp task @xcite @xcite . these models are better at capturing the context of the target domain. however, the efficacy of the dsm-based approach for addressing the covid-19 misinformation detection problem has not also been investigated. in this paper, we conduct a systematic extensive study to understand the scope and limitations of the mixed-domain transfer learning approach as well as the dsm-based approach to detect covid-19 misinformation on social media. we use both shallow and deep ptms for the mixed-domain transfer learning experimentations. the deep ptms include bert, xlnet, and two variants of bert, i.e., roberta @xcite and albert @xcite . while these attention mechanismbased transformer models are good at learning contextual representations, their ability to learn global relationships among the words in the source text is limited @xcite . the dsms used in our study are based on shallow architectures. we argue that shallow architectures can be trained efficiently using the limited available domain data. specifically, we pre-train the dsms using the small social media data on covid-19. the shallow dsm-based approach is examined in two dimensions: graph-based dsm and non-graph dsm. the graph-based text gcn @xcite model can explicitly capture the global relationships (from term co-occurrence) by leveraging the graph structure of the text. it creates a heterogeneous word document graph with words and documents as nodes for the whole corpus, and turns document classification problem into a node classification problem. we include another graphbased model in our study, i.e., the vgcn-bert @xcite . it combines the strength of text gcn (to capture global relationships) and bert (to capture local relationships). the non-graph dsm models such as word2vec and glove can mainly capture local relationships among the words of the source text, represented in the latent space of their word embeddings. for extracting global relationships from these embeddings, we utilize a convolutional neural network (cnn). specifically, we use the word embeddings as input features to a cnn with a one-dimensional kernel @xcite , which then learns global relationships as high-level features. we hypothesize that the local and global relationships should improve the generalization capability of the non-graph dsm+cnn approach. we evaluate the generalizability of the abovementioned diverse array of nlp techniques via a set of studies that explore various dimensions of the covid-19 data. we focus on the twitter social media platform because of its highest number of news-focused users @xcite . in addition to analyzing the tweet messages, we use online news articles referred to in the tweets. our study spans along multiple dimensions of the covid-19 data that include temporal dimension (the context in the dataset evolves), length dimension (short text such as tweets vs. lengthy text such as news articles), size dimension (small dataset vs. large dataset), and classification-level dimension (binary vs. multi-class data). contributions. we design a novel study for examining the generalizability of a diverse set of deep learning nlp techniques on the multi-dimensional space of covid-19 online misinformation landscape. our main contributions are as follows. the rest of the paper is organized as follows. in section 2, we present the diverse nlp techniques, analyze the multi-dimensional datasets, and describe the study design. results obtained from the experiments are provided in section 3 followed by a detailed analysis. section 4 presents the conclusion. appendix provides related work, additional analysis of the datasets, and experiment setting. ## method first, we describe how we obtained various ptms and created dsm embeddings for different models as well as how we fine-tuned/trained the classifiers for the studies. then, we discuss the datasets and the study design. ## results and analysis we evaluate the performance of the models based on the accuracy, precision, recall, and f1 score, with an emphasis on the misinformation class. for each experiment, we average the results for 10 runs. the experiments are done using scikit-learn @xcite , tensorflow 2.0 @xcite , and pytorch @xcite libraries. for creating the word2vec embeddings, we used the skip-gram model from the gensim library @xcite . finally, the glove embeddings are created using the model from (glove-python, 2016). from the results on the coaid tweets, given in figure 2 shows the results obtained from study 5. we see that the mixed-domain transfer learning approach performs poorly on the fine-grained dataset. the only model that achieves decent performance is the non-graph dsm word2vec with cnn. analysis. based on the results obtained from the studies, we summarize our observations below. first, we discuss a model's generalizability for binary classification scenarios. given the length of the text and the size of the dataset, we identify 4 cases. case 1: @xmath0, we do not find a single best-performing model. for the tweet dataset, the following models perform slightly better: vgcn-bert, glove (transfer learning and dsm-based), and text gcn. there are two possible explanations for the poor performance of all models on the short-length tweet data. first, the number of test misinformation samples is significantly smaller. for example, in the 2020 july, september, and november tweet test datasets, the true information samples are larger than the misinformation samples by 46, 17, and 96 times, respectively. second, the short length of the text and the small size of the training set might have influenced the scope of the context learning by the models. case 2: @xmath1, the best model is dsm word2vec+cnn for the july and november datasets. it achieved the highest precision and recall on the misinformation class. for the september dataset, the elmo outperforms dsm word2vec+cnn. case 3: @xmath2. however, bert's performance is not consistent. on the tweet dataset (short-length text), the precision of bert is poor. it indicates that even with larger training data, bertbased transfer learning does not provide an effective solution for short-length samples. one possible reason is that although bert is good at capturing the local relationships (e.g., word order), it does not do equally well on capturing the global relationships from short-length data. the results from fine-grained classification show that dsm word2vec+cnn outperforms other approaches by a large margin. apart from the case of binary short length and small size dataset, dsm word2vec+cnn is shown to achieve the most effective solution. one possible reason for the better generalization capability of the non-graph dsm+cnn-based approach is that its hierarchical feature-extraction mechanism is conducive for learning both the local context (the non-graph dsm, e.g., word2vec captures the local relationships of words in the target text) and the global context (the cnn learns global relationships from the word embeddings), which validates our hypothesis. based on the insights garnered from the above analysis, we draw the following conclusions, summarized in the framework in figure 3 . the vgcn-bert that combines the benefits of text gcn with bert is not effective either. our study suffers from some limitations. the lack of labeled data narrowed the scope of our investigation. the data scarcity affected our study in two ways. first, due to the small size of the test data, we obtained noisy estimates for the short length and small size data. second, we could not conduct a multi-dimensional study on the finegrained classification problem. ## conclusion when an unanticipated pandemic like covid-19 breaks out, various types of misinformation emerge and propagate at warp speed over online social media. for detecting such misinformation, nlp techniques require to capture the context of the discourse from its evolving narrative. we argue that irrespective of the success of the deep learning based mixed-domain transfer learning approach for solving various nlp tasks, it does not yield a generalizable solution. we emphasize the importance of learning the context (both local and global) directly from the target domain via the dsm-based approach. a feasible way to implement a dsm is to utilize shallow neural networks that capture the local relationships in the target data. representations learned from this type of model can then be used by shallow cnns to learn global relationships as high-level features. thus, a combination of non-graph dsm and cnn may lend a more generalizable solution. we perform an extensive study using twitter-based covid-19 social media data that includes tweets and news articles referred to in the tweets. our investigation is performed along the following dimensions of the data: temporal dimension (evolving context), length dimension (varying text length), size dimension (varying size of datasets), and classification-level dimension (binary vs. multi-class data). we show that the mixed-domain transfer learning approach does not always work well. we found the combination of the non-graph dsm (for capturing local relationships) and cnn (for extracting global relationships) to be a promising approach towards creating a generalizable solution for detecting covid-19 online misinformation. in the future, we plan to investigate the generalizability of the dsm models created using deep learning architectures such as bert. model was combined with two-dimensional maxpooling in @xcite for capturing text features. in addition to this type of chain-structured lstm, tree-structured lstm such as the tree-lstm model was developed for learning rich semantic representations @xcite . irrespective of the progress harnessed by rnn-based models, they do not perform well in capturing global relationships (i.e., long-term dependencies) among the words of the source text. also, training this type of model on large data is inefficient. an efficient approach for some nlp tasks such as text classification is a shallow convolutional neural network (cnn) with a one-dimensional convolutional kernel @xcite . this model is good at capturing local patterns such as key phrases in the text. however, it does not work effectively if the weights of the input layer are initialized randomly @xcite . it was shown to be effective only in transfer learning in which, first, word embeddings are created using a self-supervised pretrained model (ptm) such as word2vec @xcite , then the cnn uses its single layer of convolution on top of the word embeddings to learn high-level representations. the use of ptms for mixed-domain transfer learning ushered in a new era in nlp @xcite . the ptms are created from general-purpose unlabeled data by using the self-supervised learning technique. in general, the ssl technique learns representations by predicting a hidden property of the input from the observable properties @xcite . two types of ptms are used in nlp: (i) ptms that are feature extractors, i.e., learn word embeddings @xcite @xcite , which are used as input to another model for solving a downstream nlp task @xcite , and (ii) ptms that learn language models and the same ptm is adapted (fine-tuned) for solving downstream nlp tasks @xcite . the feature extractor ptms such as word2vec @xcite , glove @xcite are based on both shallow and deep neural network architectures. while shallow word2vec and glove models learn non-contextual word embeddings from unlabeled source data, the deep elmo model is good for creating contextual embeddings. features learned from these ptms are used as input to another neural network for solving a downstream nlp task using labeled target data. both word2vec and glove learn word embeddings from their co-occurrence information. while word2vec leverages co-occurrence within the local context, glove utilizes global word-toword co-occurrence counts from the entire corpus. word2vec is a shallow feed-forward neural network-based predictive model that learns embeddings of the words while improving their predictions within the local context. on the other hand, glove is a count-based model that applies dimensionality reduction on the co-occurrence count matrix for learning word embeddings. these two models are good at capturing syntactic as well as semantic relationships. although they can capture some global relationships between words in a text @xcite , their embeddings are context-independent. thus, these two models are not good at language modeling. a language model can predict the next word in the sequence given the words that precede it @xcite , which requires it to capture the context of the text. the deep architecture feature extractor ptm elmo (embeddings from language models) @xcite ) learns contextualized word embeddings, i.e., it maps a word to different embedding vectors depending on their context. it uses two lstms in the forward and backward directions to encode the context of the words. the main limitation of this deep ptm is that it is computationally complex due to its sequential processing of text. thus, it is prohibitively expensive to train using a very large text corpus. another limitation of this model, which also applies to feature extractor ptms in general, is that for solving downstream nlp tasks we need to train the entire model, except for the input embedding layer, from scratch. the above two limitations of the feature extractor ptms are addressed by a very deep architecturebased transformer model @xcite . unlike the feature extractor sequential ptms, transformer is a non-sequential model that uses self-attention @xcite to compute an attention score for capturing the influence of every word on other words in a sentence or document. this process is parallelized, which enables training deep transformer models efficiently using very large text corpus such as wikipedia and book-corpus @xcite as well as web crawls @xcite . there are two main types of transformer-based deep ptms: autoregressive and autoencoding. the openai gpt (generative pre-training) @xcite ) is an autoregressive model that learns embeddings by predicting words based on previous predictions. specifically, it is a unidirectional model that predicts words sequentially in a text. on the other hand, bert @xcite utilizes the autoencoding technique based on bi-directional context modeling. specifically, for training, it uses a masked language modeling (mlm) task. the mlm randomly masks some tokens in a text sequence, then it predicts the masked tokens by learning the encoding vectors. variants of bert such as roberta @xcite and albert @xcite were proposed to improve its effectiveness as well as efficiency. roberta (robustly optimized bert) improves the effectiveness of bert by using several strategies that include the following. it trains the model longer using more data, lengthy input, and larger batches. it uses a dynamic masking strategy and removes bert's next sentence prediction (nsp) task. albert (a lite bert) improves the efficiency of bert by employing fewer parameters, which increases its training speed. there have been attempts such as in xlnet @xcite to integrate the strengths of the autoregressive and autoencoding transformer techniques. xlnet is an autoregressive model that uses a permutation language modeling objective. this allows xlnet to retain the advantages of autoregressive models while leveraging the benefit of the autoencoding models, i.e., to capture the bi-directional context. irrespective of the state-of-the-art (sota) performance of the deep ptm based mixed-domain transfer learning approach on many nlp tasks @xcite , this approach is not suitable for detecting misinformation from covid-19 social media data. it generalizes poorly when the domain of the source dataset used to create the ptms is significantly different from that of the target dataset @xcite . one solution to this generalizability problem is to create a ptm using data that shares context similar to the target domain, i.e., pretrain a domain-specific model (dsm). this type of model encodes the context of the target domain more effectively to provide a generalizable solution for the downstream task @xcite @xcite . however, pre-training a deep architecture-based dsm (e.g., bert) for the covid-19 misinformation detection task in a timely fashion could be infeasible as it requires collecting a large amount of covid-19 social media data, which must cover the diverse landscape of covid-19 misinformation. while there was an effort to create such a deep dsm using covid-19 tweets in @xcite , capturing the dynamic context of the pandemic requires the collection of various types of social media data at a large scale. thus, to create dsms for the covid-19 domain using quickly collectible small data, shallow architecture based ptms such as word2vec and glove are suitable. however, as mentioned earlier, these ptms are context-independent and are not good at capturing the global relationships well. to compensate for these shortcomings, we used the extracted features from the word2vec and glove dsms for training a one-dimensional convolutional kernel-based cnn similar to the shallow architecture given in @xcite . the cnn learns global relationships by extracting local patterns in a hierarchical fashion by convolving over the word embeddings. another type of dsm we used is graph-based that leverages the linguistic-aware graph structure of the text for learning contextual representations, then uses those representations to solve a downstream nlp task. the main intuition driving the graph-based technique is that by modeling the vocabulary graph, it will be possible to encode global relationships in the embeddings. text gcn (text graph convolutional network) @xcite is a graph-based model that explicitly captures the global term-co-occurrence information by leveraging the graph structure of the text. it models the global word co-occurrence by incorporating edges between words as well as edges between a document and a word. word-word edges are created by using word co-occurrence information and word-document edges are created by using word frequency and word-document frequency. its input is a one-hot vector representation of every word in the document, which is used to create a heterogeneous text graph that has word nodes and document nodes. these are fed into a two-layer gcn (graph convolutional network) @xcite that turns document classification into a node classification problem. although text gcn is good at convolving the global information in the graph, it does not take into account local information such as word orders. to address this issue, text gcn was combined with bert, which is good at capturing local information. the resulting model is vgcn-bert @xcite . bert captures the local context by focusing on local word sequences, but it is not good at capturing global information of a text. it learns representations from a sentence or a document. however, it does not take into account the knowledge of the vocabulary. thus its language model may be incomplete. on the other hand, text gcn captures the global vocabulary information. the vgcn-bert aims to capture both local and global relationships by integrating gcn with bert. both the graph embeddings and word embeddings are fed into a self-attention encoder in bert. when the classifier is trained, these two types of embeddings interact with each other through the self-attention mechanism. as a consequence, the classifier creates representations by fusing global information with local information in a guided fashion.