UMUT eam at S em E val-2025 Task 7: Multilingual Fact-Checked Claim Retrieval with XLM - R o BERT a and Self-Alignment Pretraining Strategy (2025)

Abstract: In today’s digital age, the rapid dissemination of information through social networks poses significant challenges in verifying the veracity of shared content. The proliferation of misinformation can have serious consequences, influencing public opinion, policy decisions, and social dynamics. Fact-checking plays a critical role in countering misinformation; however, the manual verification process is time-consuming, especially when dealing with multilingual content. This paper presents our participation in the Multilingual and Crosslingual Fact-Checked Claim Retrieval task (SemEval 2025), which seeks to identify previously fact-checked claims relevant to social media posts. Our proposed system leverages XLM-RoBERTa, a multilingual Transformer model, combined with metric learning and hard negative mining strategies, to optimize the semantic comparison of posts and fact-checks across multiple languages. By fine-tuning a shared embedding space and employing a multiple similarity loss function, our approach enhances retrieval accuracy while maintaining efficiency. Evaluation results demonstrate competitive performance across multiple languages, reaching 25th place and highlighting the potential of multilingual NLP models in automating the fact-checking process and mitigating misinformation spread.

## introduction the massive and continuous dissemination of information on social networks makes it increasingly difficult to distinguish between accurate content and misinformation @xcite . this challenge is exacerbated by the fact that fake news can spread rapidly and influence public opinion, social behavior, and even political processes. ensuring the veracity of online content is therefore critical to safeguarding societal well-being. manual fact-checking is a slow and resourceintensive task, especially in a multilingual context where claims and their verifications may appear in different languages. to address this challenge, recent research has proposed decomposing the process into subtasks that can be partially automated using natural language processing (nlp), such as identifying check-worthy claims, retrieving relevant evidence, and assessing veracity @xcite . despite progress, automated fact-checking remains difficult due to linguistic ambiguity, implicit or biased wording, and the challenge of assessing the reliability of sources. nevertheless, automating this process is essential to combat misinformation at scale and support a more informed and resilient society. the multilingual and crosslingual fact-checked claim retrieval shared task (semeval 2025) @xcite focuses on the efficient identification of claims that have already been fact-checked in a multilingual and cross-lingual context. given a social media post, the goal is to determine the most relevant corresponding to fact-check for the post. it is divided into two "subtasks", that is, the task could be done following two different setups: (1) setup a: monolingual. in which both posts and fact-checks are in the same language; and (2) setup b: crosslingual. in which posts and factchecks may be in the same language or in different languages. for this task, we propose an approach based on xlm-roberta, a multilingual transformerbased model, to generate contextual embeddings for both social media posts and fact-checked claims, enabling semantic comparisons across languages. our approach employs a metric learning pipeline where the model is trained to optimize the cosine similarity between embeddings of relevant claimpost pairs, facilitating effective cross-lingual retrieval. to improve the model's discriminative ability, we incorporate a hard negative mining strategy that dynamically selects challenging pairs that are semantically similar but do not correspond to the same fact-checked claim. this forces the model to learn more refined feature representations, improving its ability to distinguish between highly related but distinct claims. by leveraging this multilingual embedding space and fine-tuning it with task-specific constraints, our approach efficiently retrieves fact-checked claims, addressing the challenge of misinformation detection in a multilingual and cross-lingual setting. ## background an automated fact-checking chain typically consists of several modules, each one in charge of performing one of the subtasks that make up the automation of the fact-checking process @xcite . generally speaking, this process begins with the detection of whether a fact should be verified or not and ends, after identifying which facts can be verified with the same information, with the extraction, from some data source, of the original source or set of evidences that supports or refutes the fact. specifically, this task focuses on previously factchecked claim retrieval (pfcr) @xcite : that is, given a text making a claim and a set of claims that have already been verified, the objective is to rank the fact-checked claims so that those that are the most relevant with reference to the given claim. following the process described above, this task would correspond to the last step of the automated fact-checking process. fact-checked claim retrieval has become an important research topic, mainly due to its potential applications in many challenging tasks such as the detection and correction of fake news and misinformation on digital platforms. this would allow to quickly identify this incorrect information, contrast it in real time, and improvements in search and recommendation systems by integrating mechanisms that prioritize verified and high quality information. since automated systems can continuously crawl and monitor news websites, social networks and other online sources, they are capable of identify new claims and information as they emerge. additionally, fact-checking requires prioritizing which claims should be checked. advanced algorithms make these decisions based on factors such as virality, potential impact and source credibility @xcite . once the information to be verified has been identified, machine learning models use historical data to pinpoint statements that may verify or contradict the fact being verified. this process consists of evaluating the similarity between each fact and previously verified facts, and assigning similarity scores to them, which significantly speeds up the extraction of verifying or contradicting statements. moreover, rapid advancements in nlp in recent years have led to the emergence of many pretrained transformer-based models. these models are trained on vast corpora of unlabeled text and, thanks to their transfer learning capabilities, they can be adapted to various tasks, which makes them an interesting option for this task. they have been shown to be effective in extracting verifications and identifying contradictions for a given fact @xcite . furthermore, pre-trained multilingual models based on transformers, such as mbert and xlm-roberta, have proven to be effective in the task of crosslingual fact-checking @xcite . trained on a large multilingual corpora, they are capable of handling multiple languages, making them capable of performing the extraction of information that verifies or contradict a fact even when the texts are in different languages. therefore, in this study, we tested a selfalignment pretraining strategy based on xlm-roberta and metric learning. our approach involves training a multilingual language model to match social media posts with relevant fact-checks by leveraging semantic representations. using hard pair mining and optimization with a multisimilarity loss function, the model learns to identify difficult to distinguish examples, improving the matching accuracy between posts and claims. we fine-tuned the model on a shared embedding space and used multisimilarityloss @xcite to refine similarity and dissimilarity relationships. ## system overview figure 1 illustrates the architecture of our system for claim identification in fact-checking publications. our approach is based on a self-alignment pretraining method similar to sapbert @xcite , which involves training a multilingual language model, such as xlm-roberta @xcite , through metric learning to match social media posts with relevant fact-checks. in this way, the trained model is able to identify seman-tic representations through a process of hard pair mining and optimization using a multiple similarity loss function. specifically, the pipeline of our system is organized into several stages, as shown in figure 1 . first, data loading and processing are performed. for this, we use the three datasets provided by the organizers: posts.csv, which contains texts of social media posts, including ocr of each post; fact_check.csv, which gathers verified facts (claims); and pairs.csv, which links posts to relevant claims. text encoding is then carried out using xlm-roberta, which generates embeddings for both posts and claims. these embeddings capture the semantic meaning of the texts in a shared vector space, facilitating similarity matching. the next step involves hard pair mining, using the multisimilarityminer library to identify examples that present high similarity in the embedding space and are therefore more difficult to distinguish, as they are not present in the pairs.csv file. once the pairs and hard pairs are identified, the model is trained using multisimilarityloss as the loss function, which adjusts the similarity and dissimilarity relationships between pairs of embeddings. additionally, adamw is employed as an optimizer to efficiently update the model parameters. figure 2 shows the pipeline used for the evaluation of the model. a previously trained model was used to identify the most relevant claims related to social media posts. in order to speed up the search time, a dictionary of embeddings of the facts was created. these claims are tokenized and passed through the trained model to obtain their representations in the form of embeddings. these representations are stored in a list and concatenated with their corresponding fact_id. once this fact dictionary is available, to extract the top 10 most related claims to an input post, the embedding of the post is obtained using the trained model and the distance between the embedding of the post and the pre-generated embeddings of the claims is calculated. from the distance matrix, the indices of the closest (i.e., most similar) facts are selected and the first 10 results are returned. ## experimental setup for experimentation, the datasets provided by the organizers were used for both training and testing. in the training process, a set of 153,742 facts, 24,431 posts, and 25,742 pairings were available, as shown in it is important to note that the facts set is considerably large, with 272,446 examples. however, with our model and approach, the search time is significantly reduced, as it is only necessary to extract the 10 embeddings of facts most related to the embedding of the input post. in contrast, many other current approaches would require a sequential search through the entire set of facts for each post, which is much less efficient. for training, xlm-roberta was used as the base model, together with multisimilarityloss as the loss function and multisimilarityminer for hard pair generation. in addition, adamw was used as optimizer, with a learning rate of 2e-5 and a weight decay of 0.01. the model was trained for a total of 20 epochs. ## results in this task of identifying the most relevant facts related to the posts, s@10 was used as the reference metric to evaluate the performance of the models. s@10 measures the proportion of posts for which at least one of the ten retrieved facts is relevant. the detailed analysis of the results, presented in analyzing the performance by language, it can be seen that in eng, the top three teams show solid results, especially pingan ai (0.916) and pali (0.904), while umuteam achieves only 0.414, indicating significant difficulties in this language. in spa, pingan ai and pali perform excellently (0.974 and 0.97, respectively), while umuteam scores only 0.42, suggesting considerable room for improvement. in tha and msa, pingan ai, pali and tifin india achieve perfect or near perfect scores, demonstrating exceptional performance. however, although umuteam shows relatively better performance in these languages (0.786 and 0.688 respectively), it still lags behind the leading teams. the lowest overall performance is observed in pol. while pingan ai and pali score 0.926 and 0.888, respectively, umuteam scores only 0.464, which is its lowest score in this language assessment. analysis of these results shows that language diversity significantly affects model performance, with generally higher scores observed for tha and msa, in contrast to eng and pol. this variability suggests that the models used may not be equally optimized for all languages. although our team (umuteam) did not achieve a high score in the ranking, our approach has notable advantages. unlike other approaches that may require separate models or language-specific settings, our approach uses a single model for all languages and achieves a hit rate above 50% for most languages. it also has a relatively short search time, making it a more efficient and scalable solution. ## conclusion our participation in the multilingual and crosslingual fact-checked claim retrieval task (semeval 2025) addressed the challenge of identifying relevant fact-checked claims for social media posts across multiple languages. we proposed a solution based on xlm-roberta, leveraging its multilingual capabilities to generate contextual embeddings that enable semantic comparisons between posts and fact-checks. by incorporating a metric learning pipeline with hard negative mining, our approach effectively distinguished between highly related but distinct claims, improving cross-lingual retrieval accuracy. the experimental results demonstrated that our system efficiently retrieved relevant claims across various languages, achieving a hit rate above 50% for most of them. notably, the model showed competitive performance in thai and malay, highlighting its effectiveness in languages with simpler grammatical structures. however, the system's performance was lower in english and polish, indicating challenges in processing complex linguistic nuances. these results underscore the impact of language diversity on model performance, suggesting the need for language-specific optimizations. despite not achieving top ranks, our unified model approach proved to be efficient and scalable, offering a significant advantage over languagespecific solutions. the system's rapid search time and ability to handle multilingual input with a single model demonstrate its potential for real-world misinformation detection applications. it is worth noting that we trained the model using the full training set provided by the organizers, without holding out a separate validation set. this decision was motivated by the retrieval nature of the task, where the goal was to optimize a embedding space to compare posts and fact-checked claims efficiently. while this choice allowed us to leverage all available data for training robust representations, it also limited our ability to perform detailed hyperparameter tuning and ablation studies. we acknowledge this limitation and consider incorporating a validation split and more granular experiments in future work. besides, we will explore domain adaptation, cross-lingual transfer learning techniques, and addressing the challenges posed by languages with complex grammatical structures. finally, we aim to apply our fact-checked claim retrieval system to domains involving political discourse @xcite and harmful or emotionally charged narratives, such as hate speech and hope speech @xcite . in this sense, integrating multilingual claim retrieval into these pipelines could help validate or challenge politically biased or manipulative statements, especially in settings where real-time verification is critical. in future work, we plan to explore domain adaptation strategies and joint modeling approaches that combine claim verification with stance and intention detection, allowing for a more comprehensive analysis of ideological and affective discourse.