T witter- COMM s: Detecting Climate, COVID , and Military Multimodal Misinformation (2022)

Abstract: Detecting out-of-context media, such as “miscaptioned” images on Twitter, is a relevant problem, especially in domains of high public significance. In this work we aim to develop defenses against such misinformation for the topics of Climate Change, COVID-19, and Military Vehicles. We first present a large-scale multimodal dataset with over 884k tweets relevant to these topics. Next, we propose a detection method, based on the state-of-the-art CLIP model, that leverages automatically generated hard image-text mismatches. While this approach works well on our automatically constructed out-of-context tweets, we aim to validate its usefulness on data representative of the real world. Thus, we test it on a set of human-generated fakes, created by mimicking in-the-wild misinformation. We achieve an 11% detection improvement in a high precision regime over a strong baseline. Finally, we share insights about our best model design and analyze the challenges of this emerging threat.

## introduction out-of-context images are a popular form of misinformation where an image is miscaptioned to support a false claim @xcite . such image repurposing is extremely cheap yet can be as damaging as more sophisticated fake media. in this work we focus on domains important for society and national security, where implications of inexpensive yet effective misinformation can be immense. specifically, we analyze multimodal twitter posts that are of significant public interest, related to topics of covid-19, climate change and military vehicles. our goal is to learn to categorize such image-text posts as pristine or falsified (out-of-context) by means of detecting semantic inconsistencies between images and text. to that end, we first collect a large-scale dataset of multimodal tweets, twitter-comms, with * denotes equal contribution. over 884k tweets. in our approach, we fuse input image and text embeddings generated by clip @xcite via an elementwise product, and train a classifier to distinguish real tweets from automatically constructed random and hard mismatches. to validate this approach and demonstrate the usefulness of the twitter-comms dataset, we report results on human-generated test data, created to mimic real-world misinformation. we discuss the results and model ablations, and provide additional insights into the challenges of this task. our dataset is publicly available at: https://github.com/giscardbiamby/twittercomms . ## related work there exist a number of large-scale twitter datasets concentrated on topics such as covid-19 @xcite or climate change @xcite . however, it remains difficult to collect labeled misinformation. researchers have collected covid-19 misconceptions on social media via manual annotation @xcite or by linking to fact checking articles @xcite . not only are these datasets small (a few thousand samples), but they focus on false claims rather than multimodal inconsistency. here, we curate social media posts that are topical and multimodal, and we demonstrate an application to misinformation detection of human-generated fakes. recent work has developed approaches for multimodal fact checking, e.g., @xcite and @xcite , who query an external knowledge base. similar to @xcite in the news domain, we use a large pretrained model that does not require an external reference set. ## experiments next, we discuss the data used for evaluation, present our approach and ablate various design choices, report results on our evaluation sets, and provide additional analysis of the task difficulty. ## conclusion in this work we tackle a real-world challenge of detecting out-of-context image-text tweets on covid-19, climate change, military vehicles topics. to approach it, we collect twitter-comms, a large-scale topical dataset with multimodal tweets, and construct corresponding hard mismatches. we design our approach based on the clip model with several important design choices, e.g. multiplying the embeddings for multimodal fusion and increasing the percentage of hard negatives in our training data. this approach substantially improves over a powerful baseline, an off-the-shelf clip model, when evaluated on human-curated in-the-wild mismatches. we hope our work and insights will benefit multimedia forensics practitioners.