# MVACLNet: A Multimodal Virtual Augmentation Contrastive Learning Network for Rumor Detection
## 4 Experiments and Analysis
### Baselines
Following this, it adaptively aggregates unimodal features and crossmodal correlations based on a learned ambiguity score between modalities, where the score is quantified by estimating the KL divergence between distributions of textual and visual features. Finally, the aggregated multimodal feature representation is input into a classifier for rumor detection. * MRAN [30]: This model first extracts multilevel textual semantic features through different encoding layers of BERT. Then, it further utilizes TextCNN to aggregate these features in layers, thus filtering out some noise while extracting important local information. The visual features are extracted through VGG-19. Afterward, it uses text/image attention blocks and cross-attention blocks to capture intramodal and intermodal associations, thereby obtaining higher-order fusion features between textual and visual modalities. Finally, the fused multimodal feature representation is used for rumor detection. ### Comparative Experiments and Analysis  
Tables 1 and 2 exhibit the detection performance of MVACLNet and all baseline models on the two real-world datasets, where the experimental results of the baseline models were  
obtained on the same datasets according to their model structure and experimental setup. It should be noted that all experimental results are from the test set. From the experimental results of unimodal models, we can observe that BERT outperforms VGG-19, which shows that textual features are more important and influential than visual features for the rumor detection task. For the comparison between unimodal and multimodal models, BERT performs better than att-RNN and EANN on the Twitter dataset, and it also obtains better performance than att-RNN, EANN, and SAFE on the Weibo dataset, which indicates the advantage of utilizing BERT to extract textual semantic features to detect rumors. However, on the whole, its performance is still lower than that of other multimodal models, which shows the limitation of using unimodal models to detect multimodal rumors. These results also confirm the effectiveness of visual content as supplementary information and the importance of designing an effective Multimodal Feature Fusion schema for multimodal rumor detection. From the experimental results of multimodal models, we can see that Spotfake and Spotfake+ outperform att-RNN, EANN, MVAE, and SAFE on the two datasets, which proves the advantage of using different pre-trained models to extract multimodal features in the multimodal rumor detection task. MCNN and SAFE achieve better performance than att-RNN, EANN, MVAE, and SAFE on the two datasets. At the same time, CAFE \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Accuracy**} & \multicolumn{3}{c}{**Rumor**} & \multicolumn{3}{c}{**Non-Rumor**} \\ \cline{3-8}  & & **Precision** & **Recall** & **F1** & **Precision** & **Recall** & **F1** \\ \hline VGG-19 & 0.596 & 0.695 & 0.518 & 0.593 & 0.524 & 0.700 & 0.599 \\ BERT & 0.706 & 0.648 & 0.540 & 0.589 & 0.715 & 0.636 & 0.673 \\ \hline att-RNN & 0.664 & 0.749 & 0.615 & 0.676 & 0.589 & 0.728 & 0.651 \\ EANN & 0.648 & 0.810 & 0.498 & 0.617 & 0.584 & 0.759 & 0.660 \\ MVAE & 0.745 & 0.801 & 0.719 & 0.758 & 0.689 & 0.777 & 0.730 \\ SAFE & 0.766 & 0.777 & 0.795 & 0.786 & 0.752 & 0.731 & 0.742 \\ Spotfake & 0.771 & 0.784 & 0.744 & 0.764 & 0.769 & 0.807 & 0.787 \\ Spotfake+ & 0.790 & 0.793 & 0.827 & 0.810 & 0.786 & 0.747 & 0.766 \\ MCNN & 0.784 & 0.778 & 0.781 & 0.779 & 0.790 & 0.787 & 0.788 \\ CAFE & 0.806 & 0.807 & 0.799 & 0.803 & 0.805 & 0.813 & 0.809 \\ MRAN & 0.855 & **0.861** & 0.857 & 0.859 & 0.847 & 0.816 & 0.831 \\ MVACLNet & **0.891** & 0.811 & **0.922** & **0.863** & **0.949** & **0.872** & **0.909** \\ \hline \hline \end{tabular} The bold numbers indicate the optimal values for the corresponding metrics.  
\end{table} 
Table 1: The performance of different methods on the Twitter dataset.