Word2winners at S em E val-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval (2025)

Abstract: This paper describes our system for SemEval 2025 Task 7: Previously Fact-Checked Claim Retrieval. The task requires retrieving relevant fact-checks for a given input claim from the extensive, multilingual MultiClaim dataset, which comprises social media posts and fact-checks in several languages. To address this challenge, we first evaluated zero-shot performance using state-of-the-art English and multilingual retrieval models and then fine-tuned the most promising systems, leveraging machine translation to enhance crosslingual retrieval. Our best model achieved an accuracy of 85% on crosslingual data and 92% on monolingual data.

## introduction the spread of misinformation on social media poses a considerable challenge for fact-checkers, who must verify claims quickly and accurately, often across different languages. in our participation in semeval 2025 task 7 @xcite , we develop systems that retrieve the most relevant fact-checked claims for a given social media post, irrespective of linguistic differences. the task is divided into two subtasks: the task uses a dataset derived from the original multiclaim dataset @xcite , which includes over 150,000 fact-checks in 8 languages (english, spanish, german, french, arabic, portuguese, thai, and malay) and social media posts in 14 languages. our approach begins with a preprocessing stage in which raw data is cleaned and summarized to reduce noise and standardize the content. next, we employ transformer-based models-including labse @xcite , gtr-t5-base @xcite , and me5 @xcite )-within a zero-shot evaluation framework to assess their ability to retrieve semantically similar claims across languages. following the initial evaluation, the most effective models are fine-tuned using the task dataset to improve both precision and recall. an ensemble strategy based on majority voting is then applied to combine the outputs of these models, thereby mitigating the impact of individual model biases. this systematic approach offers a robust methodology for retrieving fact-checked claims in a multilingual context and addresses the challenges associated with misinformation detection. our experiments revealed that our ensemble approach consistently enhanced retrieval performance as measured by success-at-10 (s@10), showing an improvement of approximately 3-5 percentage points over the best single-model baseline. at the same time, a closer examination of the results indicates that the system tends to struggle with informal language, ambiguous expressions, and idiomatic usage, particularly in low-resource languages. these observations suggest that further refinement in preprocessing and model adaptation may help address the inherent variability and noise 1370 in social media content. ## related work previously fact-checked claim retrieval (pfcr) is the task of retrieving the most relevant factchecked claims from a dataset regarding a given social media post. each claim in the dataset has been reviewed by experts and labeled as either misinformation or not. by ranking and retrieving the most relevant claims, we can infer a label for the post based on its similarity to existing claims. for each social media post, the system is expected to retrieve 10 most related claims. the evaluation metric is success@k which is defined as the proportion of cases where a relevant item appears within the top k results returned by a system. a model is considered successful if it includes the correct label within its top 10 ranked responses. the benchmark includes early contributions from @xcite , which supported a limited number of languages and primarily focused on specific topics, such as covid-19. their dataset originated from whatsapp, containing 650 postclaim pairs. despite its limitations, it established a growing research area, inspiring later datasets to expand and refine its scope. for our study we used multiclaim, the most comprehensive dataset in the benchmark. multi-claim is multilingual, covering 39 languages and a wide range of topics while incorporating diverse cultural and social perspectives. it is collected from facebook, twitter, instagram, and google fact check tools, ensuring broad coverage and diversity. with over 31,000 post-claim pairs, mul-ticlaim serves as an excellent resource for training models and enhancing fact-checking systems. the semeval task consists of two tracks: monolingual and crosslingual. in the monolingual setting, both the post and the claim are in the same language. in contrast, the crosslingual setting involves a post and a claim in different languages, introducing additional challenges for the system. the differences between these tracks and how their respective models operate are illustrated in figure 1 . ## results in the summarization phase, we employed several llms to reduce text length and eliminate noise. the models used included mt5-multilingual-xlsum @xcite , bart-large-cnn @xcite , falcon3-7b-instruct (team, 2024b), qwen2.5-1.5b-instruct, and qwen2.5-7b-instruct (team, 2024a), with the latter demonstrating the best performance. we used the last two models to summarize the input text, and the results are presented in a wide range of bi-encoder models can be used for information retrieval, each designed to extract embeddings from the input text. by computing the cosine similarity between embedding pairs, we can determine how similar they are. to identify the most effective models, we conducted zero-shot experiments with several candidates, testing multilingual models on the original dataset as well as its english translation. interestingly, using the english translation typically lowers monolingual accuracy. this is because multilingual models are designed to generalize across multiple languages rather than being optimized for english specifically. however, translating the input text improves crosslingual accuracy. in crosslingual scenarios, where the post and claim are in different languages, multilingual models struggle to generate similar embeddings for semantically equivalent content across languages, leading to better differentiation after translation. we also tested english-specific models, which outperformed multilingual models. since these models are trained exclusively on english data, they yield the best results when applied to the translated dataset. overall, multilingual models offer better generalization across languages, while english models excel in single-language tasks. the results of our experiments are presented in tables 3 and 4 . after conducting multiple zero-shot experiments, we selected the best-performing models and finetuned them on the training dataset. the multilingual models were trained on the various languages present in the dataset, while the monolingual models were trained on its english translation. the results are shown in tables 5 and 6 . fine-tuning significantly improved model performance, highlighting its role in adapting to dataset-specific patterns. figure 2 illustrates this effect in both monolingual and crosslingual settings, demonstrating substantial improvements across models-except for the uae-large-v1 model, which overfit rapidly and failed to generalize to test data. notably, fine-tuning benefited the crosslingual setting more than the monolingual one. this is figure 3: the best model's performance (s@10) on different languages while a single model may achieve the highest overall performance, it does not necessarily produce the best results across all languages and scenarios. to address this, a voting mechanism can be employed to leverage the strengths of multiple models, leading to more robust and accurate predictions. the results of this approach are presented in ## conclusion in this study, we developed a system for retrieving fact-checked claims from a multilingual dataset. the system uses several stages, including data preprocessing, summarization, zero-shot evaluation, fine-tuning, and an ensemble majority voting approach to match social media posts with relevant fact-checks. our experiments show that fine-tuning improves performance, especially in crosslingual settings when combined with machine translation. the ensemble strategy also helps to overcome the limitations of individual models, leading to high accuracy in both crosslingual and monolingual tasks. however, the results reveal some challenges, such as managing informal language and ambiguous expressions, particularly in low-resource languages. future work should focus on enhancing preprocessing methods, exploring alternative summarization techniques, and incorporating more language resources to improve overall system robustness. overall, this study provides a clear and effective framework for fact-checked claim retrieval, which is essential for addressing the spread of misinformation.