From Generation to Detection: A Multimodal Multi-Task Dataset for Benchmarking Health Misinformation (2025)

Abstract: Infodemics and health misinformation have significant negative impact on individuals and society, exacerbating confusion and increasing hesitancy in adopting recommended health measures. Recent advancements in generative AI, capable of producing realistic, human-like text and images, have significantly accelerated the spread and expanded the reach of health misinformation, resulting in an alarming surge in its dissemination. To combat the infodemics, most existing work has focused on developing misinformation datasets from social media and fact-checking platforms, but has faced limitations in topical coverage, inclusion of AI-generation, and accessibility of raw content. To address these gaps, we present MM-Health, a large scale multimodal misinformation dataset in the health domain consisting of 34,746 news article encompassing both textual and visual information. MM-Health includes human-generated multimodal information (5,776 articles) and AI-generated multimodal information (28,880 articles) from various SOTA generative AI models. Additionally, We benchmarked our dataset against three tasks—reliability checks, originality checks, and fine-grained AI detection—demonstrating that existing SOTA models struggle to accurately distinguish the reliability and origin of information. Our dataset aims to support the development of misinformation detection across various health scenarios, facilitating the detection of human and machine-generated content at multimodal levels.

## introduction health misinformation refers to information that is inaccurate, misleading, or false according to the best available health evidence at the time. such misinformation tends to spread at unprecedented speed and scale on the world wide web and social media (office of the surgeon general (osg), 2021), and has been shown to have significant negative effects on society @xcite @xcite @xcite . during crises, such as outbreaks of infectious diseases like covid-19, the overproduction of health misinformation in both digital and physical environments is defined as an infodemic. as society transitions into a long covid era @xcite , the infodemic continues to evolve, leading to a reduction in public trust in health professionals @xcite . furthermore, unfiltered exposure to health misinformation can delay or prevent effective disease treatment and even threaten the lives of individuals @xcite . additionally, the recent advent of generative image models, such as stable diffusion @xcite , mid-journeyv5 @xcite , dall-e2 @xcite , as well as human-level text generators like chatgpt @xcite and llama @xcite , has amplified both the quantity and spread of misinformation @xcite . the ease with which generative ai models can replicate or manipulate multimodal content, including text and images @xcite , further contributes to the emergence of health misinformation @xcite @xcite . this presents a significant new challenge in combating the infodemic. recent studies have focused on developing and analyzing datasets and detection methods to combat the infodemic. common methods for collecting multimodal health misinformation include scraping social media platforms @xcite @xcite @xcite @xcite , such as twitter (now known as x), and parsing fact-checking websites @xcite @xcite , such as snopes and poynter. however, these datasets exhibit the following notable limitations: the majority of existing datasets focus solely on human-generated misinformation, ignoring the growing trend of ai-generated misinformation @xcite @xcite . with ai models now capable of generating and manipulating both text and image content @xcite , it is crucial to include ai-generated misinformation to ensure data diversity. the limitation of the previous health misinformation datasets are summarised in ## related work web and social media provide valuable sources of information and play important roles in multiple tasks like depression and suicide identifica-tion @xcite , health mention classification @xcite and creditable knowledge management @xcite . however, systematic reviews shows that health-related misinformation on web and social media posts critical threat to the community, which could lead to delay or prevent treatment and even threaten the lives of individuals @xcite . ## mm-health dataset the overall data collection process for mm-health, including human and ai-generated content, is illustrated in figure 1 . this process can be summarised as follows: 1) utilising multiple existing open-source health misinformation datasets as data sources, 2) validating the available data samples and collecting human-generated multimodal data from the provided urls, and 3) implementing generative ai models to collect ai-generated replicated multimodal data. to ensure data quality, both human and ai generated content are evaluated by five human evaluators proficient in english. ## experiments existing multimodal vision llms can provide insightful descriptions for input text and image data, offering the most accessible way to interact compared to other deep learning models that require extensive training. we utilise both proprietary and open-source models as baselines to detect data reliability and originality using our mm-health dataset and describe the benchmarking of these models. ## case study to provide a deeper understanding of our quantitative results, we conducted a qualitative analysis of representative classification errors. ## conclusion we introduce mm-health, a multimodal dataset designed for detecting human and ai generated health misinformation. unlike datasets limited to human generated information or augmented with llm generated text, our approach combines text and image generative models to create multimodal counterparts of misinformation. to ensure diversity in generative model outputs, we incorporate five different open-source models for each modality. our experiments involve six different vllms, including proprietary gpt-4o and gpt-4omini models, as well as sota open-source models, to evaluate data reliability and originality classification. experimental results reveal that current vllms cannot accurately detect misinformation and aigenerated content, highlighting challenges in the era of generative ai. we hope publicly releasing the mm-health dataset guides future research.