# Global Local Fusion Neural Network for Multimodal Sentiment Analysis
## 4 Experiments
### Ablation Studies  
To verify the effectiveness of each proposed module of GLFN, we carry out ablation experiments on the MVSA dataset. We remove textual global fusion learning (wo-TGF), visual global fusion learning (wo-IGF), textual local fusion learning (wo-TLF), and visual local fusion learning (wo-ILF) of GLFN to evaluate the influence of each part. In addition, we further investigate the importance of the cross-modal fusion module and fine-grained fusion module of local fusion learning, removing the cross-modal fusion module (wo-CM) and fine-grained fusion module (wo-FF) independently. The results of the ablation experiments are reported in Table 2.  
From the results, we can observe that the full version of GLFN achieves the best results; removal of either part of the model will affect the performance. This indicates that all these parts are adequate for the sentiment task. For global fusion learning, textual fusion features are more important than visual features for the prediction. The importance of the sentiment of visual and textual local fusion features is different by the datasets, which strongly depend on the relationship between images and texts. The additional investigation for local fusion shows that the cross-modal fusion module is more important than the fine-grained fusion module, and the fine-grained fusion further enhances the performance of the model. Compared with other parts, textual global fusion features are more effective for the prediction.  
We also consider the number of region proposals in the image as a hyperparameter that affects the interaction between image and text, and the performance of sentiment analysis. In addition, we conduct experiments with different top \(k\) region proposals where objects exist. Figure 4a,b show the experimental results of accuracy and F1-score. MVSA-S represents the results obtained from MVSA-Single, and MVSA-M denotes the results performed on MVSA-Multiple. The figures show that when \(k=3\), both the accuracy and F1-score reach the maximum values on the two datasets. The values show that correlation discovery between image regions and words can be found within three region proposals in the image, and the local learning is more valid when the number of region proposals equals three. Therefore, for all the experiments reported in this paper, the region number is set to three. Here, \(k=0\) denotes the local fusion learning is removed from the model, which has a significant influence on the performance. The results indicate the effectiveness of local fusion learning.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**MVSA-Single**} & \multicolumn{2}{c}{**MVSA-Multiple**} \\ \cline{2-5}  & **ACC\(\uparrow\)** & **F1\(\uparrow\)** & **ACC\(\uparrow\)** & **F1\(\uparrow\)** \\ \hline wo-TGF & 0.7442 & 0.7366 & 0.7398 & 0.7189 \\ wo-IGF & 0.7597 & 0.7497 & 0.7403 & 0.7268 \\ wo-TLF & 0.7637 & 0.7517 & 0.7396 & 0.7218 \\ wo-ILF & 0.7597 & 0.7497 & 0.7430 & 0.7281 \\ wo-CM & 0.7580 & 0.7522 & 0.7428 & 0.7253 \\ wo-FF & 0.7641 & 0.7558 & 0.7437 & 0.7293 \\ \hline GLFN & 0.7721 & 0.7642 & 0.7587 & 0.7399 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Ablation study results of the GLFN model (\(\uparrow\) denotes the value is larger, the performance of model is better).