# Multi-Task Supervised Alignment Pre-Training for Few-Shot Multimodal Sentiment Analysis
## 4 Experiments
### Experimental Results  
In our experiment, we compare multiple multimodal model methods and further evaluate their performance according to two metrics: standard accuracy (Acc) and weighted F1 measure (Weighted-F1). Among other multimodal model methods, MVAN [3] and MGNNS [34] are multimodal fusion methods and LXMERT [35] is a classic pre-trained multimodal model. PF [36] and LM-BFF [37] employ prompt learning for texts. PVLM [5] directly feeds vision features into the PLM for multimodal prompt learning. UP-MPF [9], \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Train\({}_{\textit{few}}\)/Train** & **Dev\({}_{\textit{few}}\)/Dev** & **Test** \\ \hline MVSA-S & 36/3224 & 36/403 & 408 \\ MVSA-M & 129/12,904 & 129/1613 & 1616 \\ TumEmo & 497/124,432 & 497/15,554 & 15,559 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Statistics on the benchmark datasets. the previous state-of-the-art model, utilizes alignment pre-training to align modalities. From the result from our experiment, as shown in Table 3, we can conclude the following:  
(1) The performance of prompt-based learning methods like PF and LM-BFF is consistently better than conventional multimodal fusion methods across the three datasets. This further proves the effectiveness of prompt-based methods, which may be attributed to the flexibility of prompt-based learning, as it can easily adapt to new tasks by tailoring prompts rather than requiring the extensive structural modifications that multimodal fusion methods may require. (2) Methods like UP-MPF utilize vision-language alignment pre-training to help the model better understand the semantic information included in the image-text pair. This, in turn, enhances performance in the FMSA task compared to other methods, except our MSAP method. (3) Our MSAP outperforms existing methods on all three datasets, achieving a new state-of-the-art performance on the FMSA task. Our MSAP approach distinguishes itself by garnering unique semantic information from a variety of tasks, along with the accomplishment of multimodal alignment between language and vision. This results in an enhanced performance in FMSA tasks as compared to the existing methods. As shown in Table 3, our MSAP method outperforms existing methods on all three datasets, achieving a new state-of-the-art performance on the FMSA task. In addition, as shown in Table 4, we comprehensively compare the performance of the MSAP method with classic baselines including UP-MPF on the fine-grained MSA task. We find that MSAP significantly outperforms existing methods on all three datasets, further verifying its effectiveness in this task. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Twitter-15**} & \multicolumn{2}{c}{**Twitter-17**} & \multicolumn{2}{c}{**MSAAD**} \\ \cline{2-7}  & **Acc** & **Weighted-F1** & **Acc** & **Weighted-F1** & **Acc** & **Weighted-F1** \\ \hline TomBERT & 44.89 (\(\pm\)3.97) & 44.73 (\(\pm\)5.70) & 47.37 (\(\pm\)0.93) & 46.59 (\(\pm\)1.26) & 73.21 (\(\pm\)1.06) & 73.31 (\(\pm\)0.61) \\ EF-CaTrBERT & 42.79 (\(\pm\)4.96) & 41.43 (\(\pm\)8.64) & 44.98 (\(\pm\)3.44) & 44.34 (\(\pm\)2.67) & 67.79 (\(\pm\)2.71) & 68.29 (\(\pm\)2.69) \\ LXMERT & 39.39 (\(\pm\)9.94) & 37.78 (\(\pm\)7.01) & 42.30 (\(\pm\)5.50) & 36.30 (\(\pm\)2.20) & 70.90 (\(\pm\)1.23) & 71.42 (\(\pm\)1.17) \\ LM-BFF & 49.68 (\(\pm\)3.62) & 47.82 (\(\pm\)1.28) & 50.76 (\(\pm\)0.84) & 50.28 (\(\pm\)0.75) & 74.67 (\(\pm\)0.51) & 75.05 (\(\pm\)0.43) \\ PF & 50.53 (\(\pm\)2.79) & 50.14 (\(\pm\)4.22) & 50.42 (\(\pm\)1.36) & 49.06 (\(\pm\)4.28) & 72.41 (\(\pm\)1.03) & 72.66 (\(\pm\)0.82) \\ UP-MPF & 56.03 (\(\pm\)3.15) & 56.41 (\(\pm\)3.41) & 52.61 (\(\pm\)2.04) & 51.96 (\(\pm\)3.03) & 75.82 (\(\pm\)0.59) & 76.19 (\(\pm\)0.58) \\ \hline
**MSAP** & **64.67 (\(\pm\)0.55)** & **63.85 (\(\pm\)0.36)** & **62.86 (\(\pm\)0.59)** & **59.52 (\(\pm\)1.71)** & **79.62 (\(\pm\)0.36)** & **78.47 (\(\pm\)0.29)** \\ \hline \hline \end{tabular}
\end{table}