# Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks
## 2 Related Work  
The overwhelming majority of the researchers of natural language processing (NLP) address their single benchmark task by engineering intermediate representations. The task-specific features are effective, but they depend on the researchers' abundant linguistic knowledge. [PERSON] _et al._[27] advocate a multilayer neural network architecture, which takes a sentence as input and extracts the features layer by layer. The first layer maps each word into a feature vector using a lookup table operation, starting from a random initialization. Subsequent layers extract local features from the sentence and feed a fixed-sized global feature vector by following standard neural network layers. They trained the neural networks by maximizing likelihood over large unlabeled data sets and then let the training algorithm discover useful representations for all types of NLP tasks. [PERSON] _et al._[28] introduced the CBOW (continuous bag-of-words) and Skip-gram models, which are efficient methods for learning high-quality word vectors from much larger unstructured text datasets. [PERSON] _et al._[29] created the tool _word2 vec_, which provides an efficient implementation of the CBOW and Skip-gram architectures for computing word vectors. In addition, [PERSON] _et al._[30] demonstrated that the word vectors capture syntactic and semantic regularities in English language. [PERSON] [7] trained a convolutional neural network on top of pre-trained word vectors and improved upon the state of the art for the task of sentence classification.  
[PERSON] _et al._[31] demonstrated that a deep convolutional neural network (DNN) is capable of achieving record-breaking results for image classification. [PERSON] _et al._[32] constructed a large-scale Visual Sentiment Ontology (VSO) and built SentiBank, a visual concept detector library for visual sentiment analysis. [PERSON]. _et al._[33] trained a DNN model named DeepSentiBank (SentiBank 2.0) on Caffe for visual sentiment concept classification. [PERSON] [10] used a pre-trained DNN [31] on the ILSVRC-2012 dataset and transferred the learned parameters to the task of visual sentiment prediction.  
However, research on combining visual and textual elements in sentiment prediction from microblogs is far behind the single-element research. [PERSON] _et al._[22] fine-tuned a CNN on a collection of images from Getty Image for visual sentiment analysis and trained a paragraph vector model for textual sentiment analysis. [PERSON] _et al._[24; 25], and [PERSON] _et al._[23] established a dataset consisting of textual messages with related images extracted from Sina Weibo and performed sentiment analysis by combing the prediction results of using n-gram textual features and mid-level visual features [32]. Our work is greatly motivated by the work performed in [7; 23; 26]. We train a CNN on word vectors pre-trained using the _word2 vec_ tool to extract features from texts, train a DNN with DropConnect [26]to learn visual features extracted from images, and then, make sentiment predictions by combining the textual and visual features.