Large Language Models for Multilingual Previously Fact-Checked Claim Detection (2025)

Abstract: In our era of widespread false information, human fact-checkers often face the challenge of duplicating efforts when verifying claims that may have already been addressed in other countries or languages. As false information transcends linguistic boundaries, the ability to automatically detect previously fact-checked claims across languages has become an increasingly important task. This paper presents the first comprehensive evaluation of large language models (LLMs) for multilingual previously fact-checked claim detection. We assess seven LLMs across 20 languages in both monolingual and cross-lingual settings. Our results show that while LLMs perform well for high-resource languages, they struggle with low-resource languages. Moreover, translating original texts into English proved to be beneficial for low-resource languages. These findings highlight the potential of LLMs for multilingual previously fact-checked claim detection and provide a foundation for further research on this promising application of LLMs.

## introduction false information spreads rapidly across digital platforms, undermining public trust and distorting public discourse, posing a significant challenge to society. in response, fact-checking has become a crucial defense against false information. however, manual fact-checking remains a slow and laborintensive process, often lagging behind the speed at which false claims propagate. to scale these efforts, automated fact-checking has emerged as a promising avenue to assist fact-checkers @xcite . a practically important but underexplored area of automated fact-checking is previously factchecked claim detection (pfcd), the task of identifying whether a given claim has already been verified (see an example in figure 1 ). misinformation often reappears in paraphrased or translated forms in different languages, leading to repeated verification of the same claims @xcite @xcite . interviews with professional fact-checkers and journalists confirm that re-verifying already fact-checked claims is inefficient, which makes previously fact-checked claim detection one of the most crucial tasks @xcite . while pfcd is crucial in combating misinformation, its challenges become even more pronounced in multilingual contexts @xcite . with hundreds of languages and independent fact-checking organizations worldwide, fact-checkers increasingly need to identify previously fact-checked claims across languages to avoid duplicating work. yet, most existing pfcd approaches are monolingual, leaving fact-checkers in low-resource languages at a disadvantage when combating false information. moreover, determining whether two claims refer to the same fact becomes increasingly challenging when they are in different languages and cultural contexts, especially in cases where high-quality data are limited. these challenges underscore the need for multilingual approaches that can operate reliably across languages and bridge the gap between multilingual misinformation and monolingual fact-checking resources. in this work, we build upon the definitions of @xcite and frame pfcd as a binary classification task to better reflect the practical needs of fact-checkers. we investigate whether large language models can reason about semantic relationships between claims and posts, aiming to improve the detection of previously fact-checked claims in multilingual and low-resource scenarios. to this end, we conducted the first systematic evaluation of multilingual llms for pfcd, treating the task as binary relevance classification. we benchmark seven open-source llms across 20 languages, covering both monolingual and crosslingual settings. 1 to evaluate the capabilities of multilingual llms on the pfcd task, we created a novel, manually annotated multilingual dataset consisting of 16k post-claim pairs labeled for relevance. in addition, we provide empirical insights into the various prompting strategies to understand how task framing affects llm performance and analyze where current models lag behind, especially in low-resource languages. our findings highlight the potential and limitations of llms, offering a foundation for future research. our contributions are twofold: most studies have focused on monolingual settings, primarily in english @xcite @xcite , limiting their applicability to other languages. recent efforts have begun exploring multilingual pfcd by developing multilingual datasets and exploring the performance of existing ir systems @xcite . @xcite created the multiclaim dataset with over 27 languages, demonstrating that english embedding models with translated data achieved superior results compared to multilingual models with input in the original language. llms for detecting previously fact-checked claims. the rise of llms has opened new possibilities for pfcd in both monolingual and multilingual settings. while most existing research relies on embedding-based similarity, only a few studies have applied llms to pfcd, and these are limited to single-language scenarios @xcite . two main strategies dominate prior llm-based approaches: textual entailment (choi and ferrara, 2024a,b), which labels claim relationships as entailment, contradiction or neutral, and generative re-ranking @xcite , which re-ranks retrieved fact-checks based on conditional probabilities. these approaches do not fully align with the needs of fact-checkers. entailment labels or ranked outputs do not always capture the broader notion of relevance. our work builds on this gap by proposing a broader framing of pfcd. we formulate pfcd as a binary classification problem, where the goal is to determine whether a fact-checked claim is relevant to a given post. our definition of relevance encompasses a wider range of relationships, including entailment, contradiction, partial overlap and generalization. unlike previous work, we define relevance less strictly without considering the stance of the post or retrieved claim: a fact-checked claim is considered relevant if it can assist in verifying the post, regardless of whether it supports or refutes the claim. this definition aligns more closely with human fact-checking workflows and emphasizes shared information useful for verification. for example, consider the post: "this is a recent photo of the peasant protests in the netherlands. german media do not report on the current protests." and the previously fact-checked claim: "this is what they don't show you port of rotterdam, thursday 18 november flat, strikes media has plenty of time for riots". while there is no direct entailment or contradiction between them, the fact-checked claim contains information relevant to verifying the post, illustrating our broader notion of relevance. see detailed explanation in appendix b. ## methodology we assess the ability of llms to determine the relevance between social media posts and previously fact-checked claims by instructing them to classify each post-claim pair as either relevant or irrelevant. our experiments consider both monolingual settings, where the post and fact-checked claim are in the same language, and cross-lingual settings, where they are in different languages. we proposed a pipeline, illustrated in figure 2 , to facilitate the evaluation by identifying fact-checked claims relevant to a given social media post. the pipeline consists of two main steps. first, the retriever component retrieves the n most similar previously fact-checked claims from a database using an embedding-based similarity (section 3.1). in the second step, an llm determines the relevance of the retrieved claims to the social media post. the role of llm is, therefore, to filter out false positives from the first stage. to validate the pipeline, we created a manually annotated dataset, where human annotators assessed the relevance between posts and retrieved claims (section 3.2). ## experiments and results this section presents overall findings on llms' performance for the pfcd task (section 4.1), followed by evaluations in monolingual (section 4.2) and cross-lingual settings (section 4.3). we also assess the impact of english translations -provided in the multiclaim dataset via the google translate api -on llms' performance (section 4.4). since the translations are provided with the dataset, we do not compare different translation models or evaluate translation quality. for calculating statistical significance, we used the mann-whitney u test @xcite for pairwise comparisons and the kruskal-wallis test @xcite for multiple groups, with significance defined as @xmath0.05. ## error analysis this section analyzes the errors in the reasoning generated by llms, focusing on the cot and xlt techniques. we categorize errors into two types. first, output consistency errors, where llm's responses are inconsistent or deviate from the expected format. second, reasoning errors, which account for misclassified relevance pairs. ## discussion multilingual previously fact-checked claim detection and low-resource languages. our experiments revealed that llms work reasonably well in english and high-resource languages, demonstrating robust capabilities in detecting previously fact-checked claims. however, a notable performance gap persists for some low-resource languages and those with non-latin scripts. this disparity emphasizes the need for tailored adaptations, particularly for non-english settings. ## conclusion this paper presents a comprehensive evaluation of seven llms, ranging from 7b up to 123b parameters, for detecting previously fact-checked claims in monolingual and cross-lingual settings. we created and released a new dataset consisting of 16,000 manually annotated post-claim pairs across 20 languages. our findings show that bigger llms perform best overall, while smaller models like qwen2.5 7b can compete effectively when combined with cot prompting. we observed consistent gaps in performance for low-resource languages and non-latin scripts. in practical settings, this suggests that translationbased approaches remain necessary to ensure robust results across languages. our findings underscore the importance of both llm selection and prompting strategies in optimizing performance for previously fact-checked claim detection tasks.