# MVACLNet: A Multimodal Virtual Augmentation Contrastive Learning Network for Rumor Detection
## 4 Experiments and Analysis
### Baselines  
To evaluate the effectiveness and superiority of MVACLNet, we compare it with two types of baseline models: unimodal models and multimodal models, where the former is based on textual or visual content in the post and the latter relies on both textual and visual content in the post. (a) Unimodal Models:  
* VGG-19 [51]: This model is a pre-trained deep convolutional neural network architecture with 19 layers, which is widely employed for image classification tasks and is known for its straightforward yet effective stacked convolutional layer structure. We used it to obtain visual feature representation, which is input into a fully connected layer followed by a softmax layer to detect rumors. * BERT [39]: This model is a pre-trained language model based on the Transformer encoder architecture, which captures bidirectional contextual textual information. We used it to obtain textual feature representation, which is input into a fully connected layer followed by a softmax layer to detect rumors. (b) Multimodal Models: * att-RNN [25]: This model employs LSTM to learn a joint representation of text and social context, and it extracts visual features through VGG-19. Then, it designs a neuron-level attention mechanism to capture correlations between visual and textual social features to obtain an attention-aggregated visual representation. Finally, it concatenates the representations for rumor detection. For fairness in the comparison, we removed the part that deals with social features in the concrete implementation. * EANN [20]: This model uses TextCNN [40] and VGG-19 to extract textual and visual features, respectively. Then, it concatenates them as a multimodal feature representation, which is input into an event discriminator and a rumor classifier. The event discriminator guides the model to capture event-invariant multimodal features through an event adversarial mechanism. * MVAE [21]: This model utilizes the variational autoencoder and a designed multimodal reconstruction loss to learn a shared representation between textual and visual modalities, where the encoder extracts textual and visual features through bidirectional LSTM and VGG19, respectively. Finally, the sampled latent multimodal feature representation is used for rumor detection. * Spotfake [23]: This model uses BERT to extract textual features and utilizes VGG19 to capture visual features. Then, it concatenates them for rumor detection. * Spotfake+ [24]: This model is an upgraded version of SpotFake that replaces BERT with the pre-trained language model XLNet [52] to extract textual features. * SAFE [22]: This model first converts an image to text through a pre-trained image2 sentence model. Then, it uses TextCNN to capture textual and visual features, which are concatenated for rumor detection. Meanwhile, it further utilizes the relevance between modalities, quantified as crossmodal similarity, to define an extra detection loss, thereby helping identify rumors. * MCNN [28]: This model applies BERT and BiGRU to capture textual semantic features and utilizes ResNet50, an attention mechanism, and BiGRU to extract visual semantic features. Meanwhile, it captures visual tampering features through the Error Level Analysis (ELA) algorithm and ResNet50. Then, it uses a crossmodal weight-sharing layer and the attention mechanism to fuse all the above features and the semantic features of ResNet50 output for prediction. Afterward, it further employs cosine similarity to measure the similarity between textual and visual semantic features, which is used to define an extra detection loss to help detect rumors. * CAFE [29]: This model utilizes BERT and ResNet34 to extract textual and visual features, respectively. Then, it defines an auxiliary correlation learning task to help achieve crossmodal feature alignment.