Sentiment Analysis of Tweets using Heterogeneous Multi-layer Network Representation and Embedding (2020)

Abstract: Sentiment classification on tweets often needs to deal with the problems of under-specificity, noise, and multilingual content. This study proposes a heterogeneous multi-layer network-based representation of tweets to generate multiple representations of a tweet and address the above issues. The generated representations are further ensembled and classified using a neural-based early fusion approach. Further, we propose a centrality aware random-walk for node embedding and tweet representations suitable for the multi-layer network. From various experimental analysis, it is evident that the proposed method can address the problem of under-specificity, noisy text, and multilingual content present in a tweet and provides better classification performance than the text-based counterparts. Further, the proposed centrality aware based random walk provides better representations than unbiased and other biased counterparts.

## introduction with the growing popularity of twitter, sentiment analysis of tweets has drawn the attention of several researchers from both academia and industry in recent times. unlike other regular texts, sentiment analysis on twitter text poses plenty of challenges because of various characteristics such as (i) under-specificity due to text limits, (ii) free-form writing such as the presence of user-defined hashtags, mentions, emoticons, (iii) noisy texts due to the presence of short-form, long-form, multilingual, transliterated text, misspelling. researchers try to address these problems by adopting various methods like task-specific representation learning @xcite @xcite @xcite , incorporating additional information such as hash-tags @xcite , relationship between users @xcite , multisource information @xcite , ensembling @xcite @xcite , etc. this paper proposes a novel approach to handle the above issues using a heterogeneous multi-layer network representation of a tweet. a multi-layer network is a network formulated by connecting different layers of networks. for example, a heterogeneous multi-layer network can be formed by connecting layers of networks of mentions, hashtags, and keywords. multi-layer networks have shown to provide promising performance in other tasks like community detection and clustering @xcite , node classification @xcite @xcite , representation learning in graphs @xcite @xcite . a tweet or a collection of tweets can be represented by a multi-layer network. an advantage of using network-based representation is that a network can be expanded by adding nodes or shrunk by removing nodes. the motivations of using a multi-layer network in this paper are as follows. (i) the semantic relation between keywords, hashtags, and mentions can be captured by applying an effective network embedding method. (ii) the noise and under-specificity can be reduced by expanding the network with related nodes or by shrinking the network after removing the unrelated nodes. further, the co-occurring keywords, hashtags, and mentions often share semantic relationships @xcite @xcite . this paper has four major contributions. first, it transforms a tweet into a multi-layer network. second, it proposes a centrality 1 aware random walk over the multi-layer network. third, it generates multiple representations of a tweet using the proposed centrality aware random walk and builds an early-fusion based neural sentiment classifier. fourth, it also addresses under-specificity and noisy text for sentiment classification by expanding or shrinking the network representing the tweets. as such, sentiment classification is a domain-dependent task @xcite . therefore, we evaluate the proposed method over datasets in different domains. from extensive experimental evaluations, the proposed method is found to outperform its counterparts in the majority of the cases. to the best of our knowledge, this study is the first of its kind to investigate sentiment classification task by transforming tweet into a heterogeneous multi-layer network. the rest part of the paper is organized as follows. section 2 presents the literature related to this study. section 3 presents the proposed framework. the experimental setup is described in section 4. the results and observations are analyzed in section 5. finally, section 6 concludes the study of this paper. ## related studies sentiment analysis is an old research area. initial work on sentiment classification can be traced back as early as 2000 @xcite @xcite . there have been several paradigm shifts in sentiment analysis methods from statistical methods @xcite @xcite to rule-based @xcite , to lexicon-based @xcite @xcite , to featurebased @xcite , to deep neural network @xcite . majority of the recent studies focus on the application of neural network models. therefore, this section briefly reviews a few of the recent and related studies which have exploited graph and neural models. authors in @xcite use a homogeneous network known as word graph to represent a document by connecting co-occurring words in the document. three different networks are created for positive, negative, and neutral classes using the documents in respective classes. using these networks, a document is represented by a threedimensional vector defined by the three sentiment classes. the elements of the vector correspond to the similarity of the word graph of the document and the word graph of the respective sentiment class. the vector thus obtained is used for classifying the document. similarly, authors in @xcite construct co-occurrence word-graph of a document collection and generate word embedding using node2vec @xcite . the embeddings thus obtained are used to represent words in the text and build a classifier using the convolution neural network (cnn) model. further, in the studies @xcite , the advantages of exploiting the relationship between keywords, sentiment, products and users have also been evident in sentiment analysis. in recent times, deep learning based models are extensively used for sentiment classification. to mention few of them, authors in @xcite @xcite @xcite use cnn, @xcite use long short term memory (lstm), bidirectional lstm (bi-lstm), @xcite use a combination of convolution and recurrent based neural network models. further, studies @xcite use neural ensemble models to combine different representation of text. ## proposed framework as mentioned earlier, the proposed method has four distinct components; (i) representation of a tweet or collection of a tweet using a multi-layer network, (ii) centrality aware random walk over the multi-layer network, (iii) tweet classification using multiple representations generated from the multilayer network of a tweet, and (iv) reduction of noise in a tweet by expanding or shrinking network. this section discusses the details of these components. figure 1 shows a high-level schematic diagram of the proposed model using a heterogeneous multilayer network. ## results and observations in even the best performances in both the metrics pertain to [c] biased rw with she embedding using both the classifiers. we feel the n2v style global topology-based biasing is not that useful for sentiment prediction than our biased approach, which uses centrality scores intuitively. among the embedding models, we observe that biased ft and she give competitive performances. we believe biased ft performs competitively as it is trained on centrality-aware random-walks, additionally augmented with sentiment polarized nodes. whereas, she systematically embeds sentiment information and also aided by biased tweet graph view -this makes it an unbeatable performer for sentiment classification. to realize the importance of generating node sequences with an effective rw method over the proposed network, we investigate another experimental setup by randomly shuffling the selected nodes for expansion (both sentiment polarized and non-polarized nodes) with the tweet text. we call it as t+shuffle-filtered and unfiltered methods for shuffling of sentiment polarized and nonpolarized node expansions respectively in @xcite . for bi-lstm, we can see @xcite unfiltered beats textonly prediction, which signifies that the list of selected nodes, though randomly shuffled, but are informative enough. for both the classifiers in [d] filtered outperforms text-only prediction on average by 0.8%, 2.4%, respectively, signifying selected nodes by sentiment polarized node expansion method aids in performance. here we shall also showcase the novelty of node sequences over a randomly shuffled list of the same nodes. [d] unfiltered is comparable with [b] view -biased rws are seen to improve upon the prior. whereas walks in the [c] view, which is comparable to [d] filtered are seen to improve the performance of the latter. ## conclusion this study investigates the efficacy of transforming tweets to heterogeneous multi-layer network for the sentiment classification task. our proposed centrality aware random-walk method can generate walk sequences that capture better semantic relations than its unbiased and biased random walk based counterparts. from various experimental observations, it is evident that sentiment-oriented node expansion can reduce under-specificity, noise in a tweet, and enhance the representation. the proposed method outperforms its text-based counterpart in a majority of the cases.