JU _ NLP at S em E val-2025 Task 7: Leveraging Transformer-Based Models for Multilingual & Crosslingual Fact-Checked Claim Retrieval (2025)

Abstract: Fact-checkers are often hampered by the sheer amount of online content that needs to be fact-checked. NLP can help them by retrieving already existing fact-checks relevant to the content being investigated. This paper presents a systematic approach for the retrieval of top-k relevant fact-checks for a given post in a monolingual and cross-lingual setup using transformer-based pre-trained models fine-tuned with a dual encoder architecture. By training and evaluating the shared task test dataset, our proposed best-performing framework achieved an average success@10 score of 0.79 and 0.62 for the retrieval of 10 fact-checks from the fact-check corpus against a post in monolingual and crosslingual track respectively.

## introduction the rapid proliferation of misinformation across social media platforms has made manual factchecking an increasingly daunting task. automated retrieval systems, powered by natural language processing (nlp) techniques, offer a scalable solution by identifying and presenting previously verified fact-checks relevant to new claims. in this work, we present a robust fact-check retrieval framework that leverages transformer-based dual encoder architectures, fine-tuned separately for monolingual and cross-lingual settings. our framework involves three state-of-the-art pre-trained models: gtr-t5 @xcite for both monolingual and coss-lingual fact-check retrieval, e5-large-v2 @xcite and minilm @xcite for cross-lingual retrieval. evaluated on the semeval 2025 shared task 7: multilingual and crosslingual fact-checked claim retrieval dataset, our proposed system achieves a success@10 score of 0.79 on the test set with gtr-t5 in the monolingual track. for the cross-lingual track, gtr-t5 and e5-large-v2 achieved success@10 scores of 0.62 and 0.58 on the test set, respectively. in addition, a minilm-based framework was also developed as a lightweight alternative that converts posts and fact-checks into normalized vector embeddings using minilm-l12-v2, which are then indexed with faiss for rapid retrieval. the proposed retrieval system not only addresses the challenge of the vast online misinformation but also provides a scalable solution that can be adapted to diverse multilingual environments. ## related work early fact-checking retrieval systems relied on keyword-based and traditional ir methods, which lacked semantic understanding and multilingual support. neural ir models like drmm @xcite and matchpyramid @xcite improved performance but struggled with scalability and cross-lingual generalization. transformer-based models such as bert @xcite and sentence-bert @xcite enabled dense vector representations and improved semantic retrieval. dual encoder models like gtr-t5 @xcite and e5 @xcite further enhanced efficiency by allowing independent query-document encoding, making them suitable for large-scale applications. earlier approaches such as dssm @xcite and knrm @xcite demonstrated the potential of deep learning for retrieval tasks but were often limited by shallow interaction architectures and difficulties in handling long input sequences. these models, while offering initial improvements over traditional ir, did not fully capture the complex semantic relationships necessary for effective claim retrieval, especially in multilingual contexts. multilingual models like labse further pushed the boundaries of multilingual semantic retrieval. ## data all textual analysis and experiments were performed using data from the semeval 2025 shared task 7: multilingual and crosslingual fact-checked claim retrieval @xcite . during the training and development phase, the dataset consisted of approximately 24,431 social media posts in multiple languages, 153,743 fact-checked claims, and 25,743 post-to-fact-check pairs where each post was linked to at least one fact-check claim. the 24,431 posts were further divided into cross-lingual and monolingual tracks where 18,907 posts were used for monolingual evaluation and 5,524 posts were used for cross-lingual evaluation. furthermore, in the monolingual track, the 18,907 posts and 153,743 fact checks were distributed into eight different languages: french (fra), spanish (spa), english (eng), portuguese (por), thai (that), deutsch (due), modern standard arabic (msa) and arabic (ara). the distribution of monolingual and crosslingual data are provided in figure 1 for both training and development sets. for the testing dataset, there was a total of 272,447 fact checks distributed over 10 languages (8 languages were the same as training and development sets, 2 extra languages were added: polish or pol and turkish or tur) and 8,276 posts. among 8,276 posts, 4000 posts were for crosslingual tracks and the remaining posts were for monolingual tracks. the overall data distribution for test data is provided in figure figure 2: distribution of test data 4 methodology this section briefly discusses the methodologies used to develop our proposed frameworks. ## result all the proposed frameworks were evaluated on the development and testing datasets using the success@10 metric by retrieving the top 10 fact-checks from the corpus. the success@10 metric can be defined as: success@10 = 1, at least one fact-chek in top 10, 0, otherwise. ## conclusion in this article, we proposed gtr-t5-based monolingual and cross-lingual frameworks and e5-large-v2 and minilm-based cross-lingual frameworks only for fact-checked claim retrieval from social media posts. our experiments show that the gtr-t5 model works well for both monolingual and cross-lingual settings with success@10 scores of 0.79 and 0.62 respectively in the test dataset. these results underscore the robustness of the proposed models in their respective tasks. however, further optimization is needed to improve recall for lower-ranked fact-checks and enhance crosslingual retrieval performance. future work will explore incorporating language mapping using multilingual transformer-based embeddings (tems) and employing advanced fine-tuning techniques to further improve performance. also, we will experiment with the e5-large-v2 and minilm models for monolingual settings in our future work.