# Global Local Fusion Neural Network for Multimodal Sentiment Analysis
## 4 Experiments
### Baseline and Comparison  
We list six studies utilizing the deep learning method for MVSA dataset as follows:  
1. MultiSentiNet [13]: employed CNN to obtain objects and scene deep semantic features of the image and utilized visual feature guided attention LSTM to extract important word features; all these features are aggregated for the sentiment analysis. 2. Co-Memory [42]: proposed an iterative co-memory model with an attention mechanism by considering the relationship between image and text; the network explored the interaction between visual and textual information multiple times to analyze users' sentiment. 3. FENet [43]: introduced an interactive information fusion mechanism, which learned the deep modality-specific fusion representation and built an information extraction network to extract information more effectively for the multimodal sentiment task. 4. MVAN [45]: proposed a multi-view attention-based interaction model, which built an iterative scene-text co-memory network, as well as an iterative object-text co-memory network to obtain semantic image-text features for the sentiment analysis. 5. CLMLF [46]: introduced contrastive learning with a multi-layer transformer-based fusion method. Two contrastive learning tasks, label-based contrastive learning and data-based contrastive learning, are proposed for training to help the model learn the common features for sentiment analysis. 6. ITIN [47]: developed an image-text interaction network to align the information between image region and words and preserved the valid region-word pairs with a cross-modal gating module for effective fusion features. The unimodal features are combined with cross-modal fusion features for the sentiment classification. Same as the previous studies, we use the accuracy rate and \(F1\)-score as the experimental evaluation metrics. The calculation is as follows:  
\[ \begin{split} Precision=\frac{TP}{TP+FP},\\ Recall=\frac{TP}{TP+FN},\\ F1=\frac{2\times Precision+Recall}{Precision+Recall},\\ Accuracy=\frac{TP+TN}{TP+TN+FP+FN}.\end{split}  \tag{40}\]  
\(TP\) is the number of samples that obtain the correct prediction with the positive label, and \(FP\) is the number of samples that wrongly indicate the prediction as positive while the label is negative. \(TN\) denotes the number of samples that are marked as negative correctly. \(FN\) presents the number with a negative prediction and positive label. The experimental results comparisons between the proposed model and the baselines are shown in Table 1. The results of baseline methods were retrieved from published papers. The results show that the MultiSentiNet obtained the worst performance among the models; even though the model considered the influence of visual information on the text, it ignored the influence of textual information on the image, and the interaction between image and text is shallow. Co-Memory considered the mutual influence and proposed an interactive fusion module to analyze the sentiment and is better than MultiSentiNet, but the coarse-grained attention mechanism may cause information loss and still need to be improved. FENet applied symmetry interactive information fusion module and the information extraction module to obtain informative representations, which separated the analysis into two sections, making the model more effective than the previous model. Since MVAN proposed a multi-view attention model to build the correlation between objects/scenes and words and integrated the features to predict results, it is slightly superior to FENet. CLMLF applied label-based and data-based contrastive learning for the sentiment task, which learned more features related to sentiment and is better than MVAN. ITIN model considered the relationship between image region and words and built deep region information based fusion features for the classification, achieving the best results among the baseline. Our model is competitive with other baseline models on the MVSA dataset.