S em E val-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval (2025)

Abstract: The rapid spread of online disinformation presents a global challenge, and machine learning has been widely explored as a potential solution. However, multilingual settings and low-resource languages are often neglected in this field. To address this gap, we conducted a shared task on multilingual claim retrieval at SemEval 2025, aimed at identifying fact-checked claims that match newly encountered claims expressed in social media posts across different languages. The task includes two subtracks: 1) a monolingual track, where social posts and claims are in the same language 2) a crosslingual track, where social posts and claims might be in different languages. A total of 179 participants registered for the task contributing to 52 test submissions. 23 out of 31 teams have submitted their system papers. In this paper, we report the best-performing systems as well as the most common and the most effective approaches across both subtracks. This shared task, along with its dataset and participating systems, provides valuable insights into multilingual claim retrieval and automated fact-checking, supporting future research in this field.

## introduction the sheer amount of disinformation on the internet is proving to be a societal problem @xcite . fact-checking organizations have made progress in manually and professionally factchecking various contents @xcite . however, manual fact-checking at scale is too costly. to reduce some of the fact-checkers' manual efforts and make their work more effective, recent studies have examined their needs and identified tasks that could be automated @xcite @xcite , such as evidence retrieval @xcite and verdict prediction @xcite . original text (german) english translation social media post wer an wahlen glaubt, ist auch der meinung, dass das ordnungsamt die küche putzt. wussten sie eigentlich das das besatzerstatut besagt: dass die fremdverwaltung allein durch die wahlen vom wahler akzeptiert wird ** bei unter 50% wahlbeteiligung wäre dies hinfällig denn es gabe dann ein rechtliches problem das besatzerstatut wäre ungültig... anyone who believes in elections also believes that the regulatory office cleans the kitchen. did you actually know that? the occupier statute states: that the foreign administration is only accepted by the voters through the elections ** if the turnout is less than 50%, this would be obsolete because then there was a legal problem the occupier statute would be invalid... ## related work the increasing prevalence of disinformation on the internet has prompted extensive research on factchecking. a number of tasks have been proposed accordingly to examine different aspects of the process @xcite , including claim detection @xcite @xcite , verdict prediction @xcite @xcite , evidence retrieval @xcite , and justification production @xcite . prior shared tasks and competitions have touched on claim verification, such as the wellknown fever shared task @xcite @xcite , and averitec shared task @xcite . evidence retrieval has also attracted significant attention. @xcite organized the shared task on evidence retrieval on clinical trial data. semeval 2020 task 9 @xcite focuses on evidence finding for tabular data in scientific documents. similarly, shared tasks are organized to tackle claim retrieval in social media posts, such as the checkthat! lab 2023 shared task @xcite , and in a multi-modal and multigenre content, such as checkthat! lab 2024 @xcite and factify 2 @xcite . ## task description the shared task is formulated as an information retrieval task. given social media posts, the goal is to retrieve the most relevant claims from a list of claims that are previously fact-checked. we set up two tracks for the shared task. in the monolingual track, matched posts and claims are always in the same language and the search pools for candidate claims are language specific to the post. in the crosslingual track, the search pool of candidate claims is not restricted and the matched claims can be in languages that are different from the post. ## baselines we selected four approaches as baselines: bm25. bm25 @xcite is used as a default retriever in many prior works on claim matching (see, e.g., @xcite @xcite and it also achieved competitive results especially in monolingual evaluation performed in @xcite . we use it with the english-translated version of the dataset. ## participating systems and results the high-level overview of approaches utilized by individual systems is summarized in ## conclusion this work presents an overview of semeval 2025 task 7 on multilingual and crosslingual factchecked claim retrieval. it provides a comprehensive analysis of the dataset and submitted systems. the task has attracted 179 participants. in the final test phase, 31 teams submitted test systems, of which 23 submitted system description papers. our analysis reveals that the most common strategy to improve base embedding models is to finetune them in a contrastive learning manner with inbatch negatives. as demonstrated by many systems, further improvements can be obtained by adding hard negatives and training on diverse subsets. a multi-stage pipeline is adopted by a large number of systems that additionally include a re-ranking and a weighted voting stage, which can bring improvements in both crosslingual and monolingual setups. an effective strategy to tackle different languages is still to translate them into english and utilize an english-focused model, but using multilingual models with the texts in original languages already gives comparable results. several systems have discovered that some techniques which work well in one track have a different impact in the other track, suggesting the difficulty of transferability. we believe our shared task along with participating systems provides valuable insights into multilingual and crosslingual claim retrieval, supporting future research in this field.