Temporal Generalizability in Multimodal Misinformation Detection (2023)

Abstract: Misinformation detection models degrade in performance over time, but the precise causes of this remain under-researched, in particular for multimodal models. We present experiments investigating the impact of temporal shift on performance of multimodal automatic misinformation detection classifiers. Working with the r/Fakeddit dataset, we found that evaluating models on temporally out-of-domain data (i.e. data from time stretches unseen in training) results in a non-linear, 7-8% drop in macro F1 as compared to traditional evaluation strategies (which do not control for the effect of content change over time). Focusing on two factors that make temporal generalizability in misinformation detection difficult, content shift and class distribution shift, we found that content shift has a stronger effect on recall. Within the context of coarse-grained vs. fine-grained misinformation detection with r/Fakeddit, we find that certain misinformation classes seem to be more stable with respect to content shift (e.g. Manipulated and Misleading Content). Our results indicate that future research efforts need to explicitly account for the temporal nature of misinformation to ensure that experiments reflect expected real-world performance.

## introduction misinformation proliferation in sectors from public health to politics has shaped public attitudes, undermining trust in reputable organizations and science as a whole. the threat of misinformation is so severe that @xcite qualifies "cyber-enabled information warfare" as an existential risk that can undermine the structure of public discourse, with its potential harm to civilization on par with climate change and nuclear warfare. although misinformation is not a novel phenomenon, its impact on society has been exacerbated by the advent of social media, which has increased the rate and ease of misinformation spread @xcite . as such, automated misinformation detection models are vital in mitigating misinformation's destabilizing effects on society. in the case of multimodal data, such as an image with a text caption, we must also consider the interaction between modalities (e.g. does the caption contradict the image?), which makes multimodal misinformation detection a harder task @xcite . nevertheless, accounting for multimodality is vital for real-world applications, since a large portion of information shared online is multimodal. while some machine learning (ml) methods can be comparable to human annotators in labelling fake news @xcite , building a multimodal model that is generalizable, explainable and scalable remains a challenge. in the current work, we explore model performance on future, unseen data (temporal generalizability). undoubtedly, the topics most subject to misinformation change rapidly with time, as does the misinformation itself. as such, a real-world model trained for optimal performance at a specific time point will likely continue to degrade in performance over time. however, most surveyed literature did not directly account for this expected drop in performance, testing models on data collected from the same time period as training data. this practice inflates expected model performance for future applications, referred to by @xcite as an issue with the model's "velocity". thus, we undertook experiments to quantify temporal generalizability of multimodal misinformation detection models. our goals are twofold: goal 1: quantify the expected drop in performance when a multimodal classifier trained on the r/fakeddit dataset is tested on out-oftemporal-domain content. goal 2: isolate the effect of content shift on the expected performance drop, specifically disentangling trends with reference to the r/fakeddit misinformation classes. ## experiment 2: multiple test splits our second experiment delved even further into goal 1. we quantified the rate of decay in model performance by increasing the number of test splits to five. we probed at the reasons for change in performance (goal 2) by comparing our resulting model (exp. 2 normal) against two variations. there are two reasons performance could change: 1. content shift; e.g. the subject of posts from apr. 2018 is different than feb. 2019 2. class distribution shift; e.g. the distribution of true vs. fake posts changes over time to isolate the effects of content shift, we evaluated on subsampled sets of the 5 test splits, enforcing the same class distribution and controlling for its effect (exp. 2 balanced). to isolate the effects of class distribution, we created a dummy classifier that predicted proportionally to class distributions observed in training. since the dummy classifier did not use content features for prediction, any observed effects over the 5 test splits reflected the class distribution's effect on performance. ## limitations our multimodal models perform worse than related works on r/fakeddit @xcite . we tried exactly recreating the architecture of nakamura et al. ( 2020 ), but model performance was still lower. noting that our investigation would benefit from being comparable to related studies, we attempted to locate the source of the discrepancy in performance but were unsuccessful. we think that our difficulty with reproducing existing results is not uncommon, and future research would benefit greatly from studies that explicitly outline guidelines on how to exactly reproduce their architecture (as in zaeem et al. ( 2020), shu et al. (2019) ). due to computational limitations, we did not fine-tune the feature extraction process on our dataset, instead relying on pretrained roberta and resnet-50. building models that are specifically tailored to misinformation datasets for feature extraction might increase performance, though it is unclear if this would change the impact of a temporal data split. future research could explore whether temporal generalizability is largely dependent on the dataset being used, and whether the obtained results would be different if another dataset was analyzed instead. additionally, we did not extensively investigate what validation strategy would work the best for temporal generalizability. instead, we naively separated the training and validation set temporally from each other, and used that for hyperparameter tuning. further work could look into training methods specifically designed for maximal temporal generalizability in misinformation detection. we study the temporal generalizability of multimodal misinformation detection models in one specific language, for one specific platform. although the experiments presented in this paper are inspired by real-world applications (e.g. deploying a misinformation detection model on a social media platform), it is worth noting that the r/fakeddit dataset contains some particularities that make it difficult to generalize to broader misinformation detection in other languages and settings. some of the samples labeled as "fake" are harmless (e.g. certain memes), although technically they are "untrue" or "manipulated". this raises the question of whether the results presented here are generalizable to datasets that focus on more "serious" topics of information (e.g. covid-19 or certain political topics). for example, some photoshopped images are evidently meant to entertain, and, although technically they constitute "misinformation", it seems unintuitive to seriously treat them as such. this raises the research question of how to effectively define "misinformation" that both makes sense semantically and also is maximally useful for automated models deployed in the real world, dealing with topics of substantial weight. ## ethical considerations we use the existing r/fakeddit dataset. since this was released as a benchmarking dataset for misinformation detection models, our use of this dataset is consistent with the use cases it was intended for. the dataset may contain personal data or offensive content so we ensure that the examples reported in this paper do not make any individuals identifiable or include offensive content. we were not able to find information on the licence associated with this dataset but since it was released for the purpose of benchmarking we assume that our use of this dataset is acceptable. we study when misinformation detection systems fail to perform well. a malicious actor could potentially exploit this knowledge to decide what kind of misinformation to spread, however, we believe that our results will be far more useful to those who are hoping to improve the temporal generalizability of their systems.