M ulti C laim N et: A Massively Multilingual Dataset of Fact-Checked Claim Clusters (2025)

Abstract: In the context of fact-checking, claims are often repeated across various platforms and in different languages, which can benefit from a process that reduces this redundancy. While retrieving previously fact-checked claims has been investigated as a solution, the growing number of unverified claims and expanding size of fact-checked databases calls for alternative, more efficient solutions. A promising solution is to group claims that discuss the same underlying facts into clusters to improve claim retrieval and validation. However, research on claim clustering is hindered by the lack of suitable datasets. To bridge this gap, we introduce MultiClaimNet, a collection of three multilingual claim cluster datasets containing claims in 86 languages across diverse topics. Claim clusters are formed automatically from claim-matching pairs with limited manual intervention. We leverage two existing claim-matching datasets to form the smaller datasets within MultiClaimNet. To build the larger dataset, we propose and validate an approach involving retrieval of approximate nearest neighbors to form candidate claim pairs and an automated annotation of claim similarity using large language models. This larger dataset contains 85.3K fact-checked claims written in 78 languages. We further conduct extensive experiments using various clustering techniques and sentence embedding models to establish baseline performance. Our datasets and findings provide a strong foundation for scalable claim clustering, contributing to efficient fact-checking pipelines.

## introduction automated fact-checking has become a crucial task to tackle the vast amount of unverified information circulating online. the core objectives of fact-checking pipelines are to identify claims that require verification, retrieve evidence, and assess their veracity automatically. the process can become even more challenging when the same claims are posted on different platforms in different languages @xcite . to overcome this challenge, the claim retrieval component in a fact-checking pipeline can retrieve, for each unverified claim, a set of previously factchecked claim matches from a database, where available @xcite . as the number of verified and unverified claims grows, performing pairwise checks of each new claim against each of the database entries becomes inefficient and impractical for scalable factchecking pipelines. since claims are often repeated, an alternative solution is to form claim clusters by grouping the verified/unverified claims discussing the same underlying facts. this not only reduces redundancy in claim retrieval and validation but also enhances the efficiency and scalability of the fact-checking process. research on identifying claim clusters has received limited attention in the literature, primarily due to the lack of suitable datasets. existing studies have applied various clustering techniques to manually verify the existence of claim clusters @xcite @xcite . however, to the best of our knowledge no prior work has assessed the quality of the clusters due to the unavailability of datasets. meanwhile, recent studies @xcite have focused on annotating claim pairs that discuss the same underlying facts, enabling a more granular analysis of relationships between two claims. however, extending this to manual annotation of groups of claims that discuss the same fact is a more challenging and time-consuming task, which has hindered creation of datasets. in this research, we address this challenge by introducing and validating a methodology for data collection and labeling, and by automatically constructing multilingual claim cluster datasets from claim-matching pairs with limited manual intervention. to the best of our knowledge, this is the first work to create dedicated datasets for claim clustering. we present multiclaimnet, a collection of three multilingual claim cluster datasets. the smaller datasets within multiclaimnet are derived from two existing claim-matching datasets, while the larger dataset is automatically constructed from the fact-checked claim dataset multiclaim @xcite . our approach for building this largest dataset involves retrieval of approximate nearest neighbors (ann) to form candidate claim pairs, followed by automated similarity annotation using three large language models (llms). mul-ticlaimnet comprises claims written in 86 unique languages across its three datasets, with the largest dataset containing 85.3k fact-checked claims. furthermore, we conduct extensive experiments on the three datasets using various clustering approaches in combination with sentence embedding models, including llms, to establish baseline performance. we make the following key contributions: ## related work claim clustering has been less widely studied than other subtasks within automated fact-checking, primarily due to the lack of available datasets. most existing studies have only explored claim clustering as a means to validate the task and to check if clusters can be found. @xcite applied a single-link hierarchical clustering, a variation of agglomeration clustering algorithm in multilingual claims to confirm the existence of claim clusters. while their study identified meaningful multilingual clusters, it did not assess the quality of the clusters identified. hale et al. (2024) followed the same approach to perform an extensive manual analysis of claim clusters found in social media posts related to the brazilian general election. to address limitations arising from the unavailability of datasets, @xcite assumed that claims related to the same article belonged to the same cluster. they employed the dbscan clustering algorithm along with a community detection algorithm @xcite to form claim clusters. the authors performed a quantitative evaluation by measuring the fraction of claims that belong to the same news article. the most relevant study, conducted by @xcite , aimed to track claim evolution across languages. the authors represented claims as vectors using sentence embeddings and identified connected components by retrieving the most similar nearest neighbors. these connected components were considered claim clusters. however, none of the works created datasets with ground-truth labels, primarily due to the challenges associated with manual annotation. instead of grouping claims discussing the same facts, several studies attempted to analyze the claims discussing the same topic. @xcite employed @xcite on sentence embeddings to obtain more granular clusters discussing the same topic. @xcite used topic modeling techniques such as lda @xcite to generate topic vectors and then applied the kmeans algorithm to form the topic clusters. in contrast to these approaches, @xcite leveraged large language models to annotate the topic associated with a claim and then trained a topic classifier in the synthetic dataset. while the topic labels can be used to form topic clusters, the number of distinct topics remains limited. ## building the claim cluster datasets we construct the claim cluster datasets from claim matching datasets, which consist of annotated claim pairs labeled as either similar or dissimilar based on whether they discuss the same underlying fact @xcite . we adhere to the definition of claim similarity provided by @xcite . if two claims are labeled as similar, we assume the similarity is bidirectional; therefore, they belong to the same cluster. this assumption enables the creation of links between similar claim pairs, forming clusters of interconnected claims. for example, if claim a is annotated as similar to claim b, and claim b is similarly linked to claim c, we infer that claims a, b, and c belong to the same cluster. we utilized two existing multilingual claim matching datasets along with automatically annotated claim pairs using large language models to form three different claim cluster datasets. ## experiment setup we employ various clustering techniques to evaluate the performance of baseline models on the claim cluster datasets. ## applications although claim clustering can be integrated across various components of automated fact-checking, we outline key applications of our datasets. ## conclusion this paper introduces multiclaimnet, a collection of three multilingual claim clustering datasets constructed from similar claim pairs. the largest dataset within multiclaimnet was created by retrieving claim pairs using approximate nearest neighbor approaches and annotating them with large language models. we conduct extensive experiments on these datasets, evaluating different clustering approaches and multilingual sentence embedding models. our results show that agglomeration clustering performs well on the largest dataset and remains competitive across the smaller ones. in contrast, hdbscan excels only on the smaller datasets, underscoring the need for robust solutions. additionally, the smaller sentence embedding models outperform larger ones, highlighting their potential for scalable fact-checking solutions. our error analysis further suggests that task-specific approaches are essential for improving clustering accuracy. we further highlight the potential applications of our dataset and believe it will be a valuable resource for advancing multilingual fact-checking research.