RACAI at S em E val-2025 Task 7: Efficient adaptation of Large Language Models for Multilingual and Crosslingual Fact-Checked Claim Retrieval (2025)

Abstract: The paper details our approach to SemEval 2025 Shared Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval.We investigate how large language models (LLMs) designed for general-purpose retrieval via text-embeddings can be adapted for fact-checked claim retrieval across multiple languages, including scenarios where the query and fact-check are in different languages. The experiments involve fine-tuning with a contrastive objective, resulting in notable gains in both accuracy and efficiency over the baseline retrieval model. We evaluate cost-effective techniques such as LoRA and QLoRA and Prompt Tuning.Additionally, we demonstrate the benefits of Matryoshka embeddings in minimizing the memory footprint of stored embeddings, reducing the system requirements for a fact-checking system.

## introduction verifying claims across various languages is becoming increasingly difficult for fact-checkers as the amount of internet content keeps growing. additionally, cross-lingual fact-checking not only requires a deep understanding of nuanced linguistic differences and diverse syntactical structures, but also demands bridging significant resource gaps in less-represented languages @xcite . recent advances in large language models (llms) have shown promise in tackling these challenges. besides generative capabilities, llms can provide high-quality textual embeddings @xcite that can be leveraged for textual retrieval @xcite . as part of semeval 2025 shared task 7: multilingual and crosslingual fact-checked claim retrieval @xcite , we explore different * phd candidate methods to adapt general-purpose retrieval llms to the downstream task of fact-checked claim retrieval. we experiment with cost-effective fine-tuning techniques, including prompt tuning @xcite , low-rank adaptation (lora) @xcite , and qlora @xcite , focusing on improving the models performance with minimal trade-offs between accuracy and resources. moreover, the amount of memory required to store fact checks in large databases grows along with the number of posts and claims on social media @xcite . to address this issue, we propose using matryoshka learning @xcite to compress fact representations while maintaining their retrieval utility and speeding up processing. our contributions are threefold: the implementation is available at https:// github.com/racai-ro/factcheckretrieval . ## related work previous work for fact-checked claim retrieval explored systems based on classical ir-models such as bm25 and semantic similarity searches using bert like models. @xcite showed that a hybrid approach results in increased performance. they proposed a two-step retrieval pipeline, comprised of a bm25 initial retrieval and re-ranking step that takes advantage of both scores from bm25 and a fine-tuned version of sentence-bert @xcite . subsequent studies @xcite ) have adopted similar approaches, incorporating semantic similarity as a standard component of the retrieval pipeline. notably, these systems were developed mainly for english, with some also addressing arabic, as the clef 2021 checkthat! @xcite challenge introduced a separate track for the language. this is a short-coming, given the universal aspect of the task. to address the language limitation and enable multilingual retrieval of previously fact-checked claims, @xcite proposed the mul-ticlaim dataset. the best results were obtained using a fine-tuned gtr-t5-large model @xcite . they observed that most embedding models tested-even those explicitly designed for multiple languages-performed better when applied to the english-translated version of the dataset. in contrast, our work focuses solely on the original multilingual data. previous approaches have primarily relied on similarity search, which in turn depends on highquality textual embeddings. @xcite demonstrated that the effectiveness of such embeddings can be significantly improved by leveraging large language models (llms) for both data augmentation and embedding generation. in line with this, bge-multilingual-gemma2 @xcite ) is a multilingual text embedding model built on the gemma2 llm architecture @xcite . trained on a wide variety of multilingual tasks, it has achieved state-of-the-art performance on the mir-acl benchmark @xcite , which is specifically designed for multilingual retrieval. this paper evaluates the effectiveness of adapting bge-multilingual-gemma2 for claim retrieval in fact-checking, focusing on parameter-efficient tuning methods. ## dataset the dataset proposed for the semeval task builds upon and extends the original multiclaim dataset @xcite . it was assembled from a va-riety of social media platforms, with post-fact pairs formed according to the fact-check alerts issued by these platforms. quantitatively, the development dataset comprises 28,092 social media posts in 27 languages, 205,751 fact-checks in 39 languages authored by professional fact-checkers, and 31,305 connections between these two categories. among these, 4,212 post-fact pairs are crosslingual. qualitatively, about 13% of the posts included multimedia attachments (image/video) and did not accurately convey the claims in text. in some of these cases, text was extracted from images using optical character recognition (ocr), but other errors may have been introduced as a consequence. the dataset remains biased toward major languages and the indo-european language family, despite its diversity. furthermore, the crosslingual pairs' applicability to other language pairs is limited because they primarily consist of posts in east or south asian languages coupled with english factchecks. ## proposed methodology in this section, we introduce our overall strategy for our retrieval system for fact-checks: contrastive fine-tuning with low-rank adaptation and matryoshka embeddings. we separately use prompt tuning to evaluate the contribution of the instruction in the prompt to the baseline's performance on the task. specifically, we investigate: ## experiments and results in this section, we present details on the training environment, followed by an overview of our experiments and the results obtained. ## conclusions and limitations in this work, we successfully adapted a generalpurpose retrieval llm for multilingual and crosslingual fact-checking through contrastive finetuning and parameter-efficient techniques, increasing its performance on the task. also, using matryoshka learning, we can significantly lower memory requirements while still achieving competitive accuracy. however, a number of limitations still exist: future work should improve information extraction from images/videos, expand the dataset, and explore multi-step retrieval methods.