Title: Towards Explainable Bilingual Multimodal Misinformation Detection and Localization
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Misinformation, Multimodal, LLM, GRPO

Abstract: The increasing realism of multimodal content has made misinformation more subtle and harder to detect, especially in news media where images are frequently paired with bilingual (e.g., Chinese-English) subtitles. Such content often includes localized image edits and cross-lingual inconsistencies that jointly distort meaning while remaining superficially plausible. We introduce BiMi, a bilingual multimodal framework that jointly performs region-level localization, cross-modal and cross-lingual consistency detection, and natural language explanation for misinformation analysis. To support generalization, BiMi integrates an online retrieval module that supplements model reasoning with up-to-date external context. We further release BiMiBench, a large-scale and comprehensive benchmark constructed by systematically editing real news images and subtitles, comprising 104,000 samples with realistic manipulations across visual and linguistic modalities. To enhance interpretability, we apply Group Relative Policy Optimization (GRPO) to improve explanation quality, marking the first use of GRPO in this domain. Extensive experiments demonstrate that BiMi outperforms strong baselines by up to +8.9 in classification accuracy, +15.9 in localization accuracy, and +2.5 in explanation BERTScore, advancing state-of-the-art performance in realistic, bilingual misinformation detection. Code, models, and datasets will be released.

TL;DR: We propose BiMi,a bilingual framework for misinformation detection with localization and cross-lingual reasoning,and introduce BiMiBench,a 104K-sample benchmark built from real news with diverse multimodal manipulations.