# Visual Enhancement Capsule Network for Aspect-Based Multimodal Sentiment Analysis
###### Abstract  
Multimodal sentiment analysis, which aims to recognize the emotions expressed in multimodal data, has attracted extensive attention in both academia and industry. However, most of the current studies on user-generated reviews classify the overall sentiments of reviews and hardly consider the aspects of user expression. In addition, user-generated reviews on social media are usually dominated by short texts expressing opinions, sometimes attached with images to complement or enhance the emotion. Based on this observation, we propose a visual enhancement capsule network (VECapsNet) based on multimodal fusion for the task of aspect-based sentiment analysis. Firstly, an adaptive mask memory capsule network is designed to extract the local clustering information from opinion text. Then, an aspect-guided visual attention mechanism is constructed to obtain the image information related to the aspect phrases. Finally, a multimodal fusion module based on interactive learning is presented for multimodal sentiment classification, which takes the aspect phrases as the query vectors to continuously capture the multimodal features correlated to the affective entities in multi-round iterative learning. Otherwise, due to the limited number of multimodal aspect-based sentiment review datasets at present, we build a large-scale multimodal aspect-based sentiment dataset of Chinese restaurant reviews, called MTCom. The extensive experiments both on the single-modal and multimodal datasets demonstrate that our model can better capture the local aspect-based sentiment features and is more applicable for general multimodal user reviews than existing methods. The experimental results verify the effectiveness of our proposed VECapsNet. Multimodal sentiment analysis; aspect-based sentiment analysis; capsule network; attention mechanism; multimodal fusion  
2022220  
Aspect-based multimodal sentiment analysis is a fine-grained task in multimodal sentiment analysis. Although aspect-based sentiment analysis on text [5; 6; 7] and multimodal sentiment analysis [8; 9] have become the hot research topics in the multimedia mining community, aspect-based multimodal sentiment analysis on user-generated reviews has received little attention. [PERSON] et al. [10] introduced the task of aspect-based multimodal sentiment analysis for the first time, and proposed a multi-interactive memory network (MIMN) to iteratively learn the text and image information for sentiment classification. [PERSON] et al. [11] proposed an adversarial training representation model for extracting the unified features of texts and images. In terms of multimodal feature fusion, the post-fusion method is adopted in most studies, which first extracts the features of each modality separately, and then concatenates them as the input of the sentiment classifier. This method uses the modal data independently to infer the sentimental polarity, thus ignoring the semantic correlation and the emotional interaction between different modalities. However, in practical applications, the data of a certain modality is often missing, or the aspect sentiment cannot be obtained from the specified one. According to the research of [PERSON] and [PERSON] [12], text is often the main carrier to convey information, especially emotional information, and images only serve as supplementary instructions rather than as an independent information source. In this regard, we design a visual enhancement network structure to address the above shortcomings. The model enhances the textual representations using the aspect-guided image features, and interactively learns the textual and visual emotional features to improve the performance of aspect-based sentiment classification. In this paper, we propose a novel visual enhancement capsule network (VECapsNet) for aspect-based multimodal sentiment analysis.