# Sentiment Interaction Distillation Network for Image Sentiment Analysis
## 3 Methods
### Sentiment Graph  
Sentiment is a intricate logical response, and the sentimental interactions among objects have a vital contribution to it. To accurately describe the sentimental interactions, we constructed a unique undirected sentiment graph (the sentimental relationships among objects) to define the interaction features. The goal of the sentiment graph is to capture interactions among sentiment-related objects, and the relationships are formulated as an adjacent matrix of sentiments. Figure 3 shows an example of a sentimental graph. Inspired by [7], we employed the panoptic segmentation algorithm as an object detector and took the objects as nodes. However, it is challenging to properly describe sentimental relations without annotation because of the gap between object semantics and sentiment. We propose utilizing the objects' semantic relationship in sentimental space as the edges.  
Given the nodes, we employ SentiWordNet [11] to label each node with sentimental polarity and strength. SentiWordNet is a lexical resource that annotates three sentiment scores: positivity, objectivity, and negativity to each synset of WordNet. We retrieved nouns and adjectives related to the node of SentiWordNet and determined the strength with the average value of related words. In particular, we defined sentimental polarity in terms of the strength of positive and negative sentiments. For example, the strength of the word \"cat\" is (0.8, 0.5), and the sentiment polarity is positive of 0.3.  
Figure 2: Overview of the Sentiment Interaction Distillation Network. During training, the object branch captures sentiment interaction information through the sentiment graph model, while the global context branch provides the context information. Then, the object information is distilled into the context feature. For testing, only the global context branch is needed for sentiment prediction.  
Based on this, we quantified the relationships among objects in terms of differences in sentimental strength. As shown in Equation (1), we took the absolute value of the difference in sentimental strength as the relationship between nodes. When the two nodes have opposite sentimental polarities, we set a greater sentimental distance to reflect the difference in sentimental relationships.  
\[A_{ij}=\left\{\begin{matrix}||S_{i}|-\big{|}S_{j}\big{|}\big{|}+1,&\quad if\ S_{i}*S_{j}>0\\ 0.5,&\quad if\ S_{i}=0,S_{j}=0\\ ||S_{i}|-\big{|}S_{j}\big{|}\big{|},&\quad otherwise\end{matrix}\right. \tag{1}\]  
On the basis of describing interactive information in the graph model, nodes in the graph model correspond to the visual features of each object. We selected handcrafted features, containing texture features and the brightness distribution chart, as the representation of objects. Inspired by [15], we observed the image intensity characteristics on the EmotionROI and the Flickr and Instagram (FI) datasets. Specifically, we quantified the brightness values of the HSI color space to 0-10 and obtained the brightness distribution chart. As shown in Figure 4, the brightness distribution can distinguish the sentimental polarity in some degree. In particular, positive sentiment has a higher distribution than negative when the brightness is 4-6, and negative sentiment is higher at 1-2. At the same time, to supplement details of the image, we utilized the Gray Level Co-occurrence Matrix (GLCM) to describe the texture feature of all objects.