# Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks
###### Abstract  
Sentiment analysis of online social media has attracted significant interest recently. Many studies have been performed, but most existing methods focus on either only textual content or only visual content. In this paper, we utilize deep learning models in a convolutional neural network (CNN) to analyze the sentiment in Chinese microblogs from both textual and visual content. We first train a CNN on top of pre-trained word vectors for textual sentiment analysis and employ a deep convolutional neural network (DNN) with generalized dropout for visual sentiment analysis. We then evaluate our sentiment prediction framework on a dataset collected from a famous Chinese social media network (Sina Weibo) that includes text and related images and demonstrate state-of-the-art results on this Chinese sentiment analysis benchmark. sentiment analysis; convolutional neural network; word vectors; microblog 1  
[PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016) [PERSON] et al. (2016)  
Many researchers have contributed to sentiment analysis of textual content or visual content. Existing methods for sentiment analysis are divided into three main categories by [PERSON] [1]: knowledge-based techniques, statistical methods, and hybrid approaches. Knowledge-based techniques such as WordNet Affect [2], SentiWordNet [3], SenticNet [4] and AffectiveSpace [5], are popular because of their accessibility and economy, but their validity depends heavily on a comprehensive knowledge base and the knowledge representation. Statistical methods have been the mainstream natural language processing (NLP) direction in research since the late 1990s. But the performance of traditional statistical text classifiers depends on a sufficiently large text input [6]. Except for the bag-of-concepts and bag-of-narratives methods in [6], recent prominent deep learning methods [7; 8; 9; 10] are also popular for their ability to analyze the sentiment of short texts by learning sentiment representations from a large corpus of labeled and unlabeled text. For example, [PERSON] _et al._[7; 8] used a convolutional neural network (CNN) to extract sentence features and performed sentiment analysis of Twitter messages. [PERSON] _et al._[9] built an ensemble system to detect the polarity of a text document from a dataset of IMDB movie reviews. CNNs have also been applied to visual sentiment analysis. [PERSON] _et al._ trained a deep CNN model called DeepSentiBank to classify visual sentiment concepts. [PERSON] _et al._[10] introduced a visual sentiment prediction framework that performs transfer learning from a pre-trained CNN with millions of parameters. Hybrid approaches [11; 12; 13; 14] exploit both knowledge-based techniques and statistical methods to perform sentiment analysis of text or multimodal data. [PERSON] _et al._[11; 12] exploited an ensemble of SenticNet and deep learning methods to infer polarity from text. [PERSON] [13; 14] built a Twitter sentiment analysis system by combining a rule-based classifier with a supervised learning method. A vast literature focuses on sentiment analysis of multimodal content. Some researchers have focused on sentiment analysis of long texts assisted by visual sentiment analysis. [PERSON] _et al._[15] centered on entity and event recognition of socially aware federated political archiving and socially contextualized broadcaster web archiving by combining opinion mining from text and multimedia. They extracted SIFT local features from images to analyze visual sentiment and assist in the sentiment analysis of text. Several researchers have performed video sentiment analysis by combining multimodal information. [PERSON] _et al._[16] performed sentiment analysis of Spanish online videos.