UWB a at S em E val-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval (2025)

Abstract: This paper presents a zero-shot system for fact-checked claim retrieval. We employed several state-of-the-art large language models to obtain text embeddings. The models were then combined to obtain the best possible result. Our approach achieved 7th place in monolingual and 9th in cross-lingual subtasks. We used only English translations as an input to the text embedding models since multilingual models did not achieve satisfactory results. We identified the most relevant claims for each post by leveraging the embeddings and measuring cosine similarity. Overall, the best results were obtained by the NVIDIA NV-Embed-v2 model. For some languages, we benefited from model combinations (NV-Embed & GPT or Mistral).

## introduction the semeval-2025 shared task 7, multilingual and cross-lingual fact-checked claim retrieval @xcite , focuses on efficiently identifying factchecked claims across multiple languages. this challenge is particularly important in the fight against global misinformation @xcite , as manual verification of claims in different languages is both timeconsuming and impractical. the task aims to support fact-checkers by developing systems that retrieve relevant, previously fact-checked claims for social media posts, addressing the complexities of a multilingual and cross-lingual context. the task utilizes an enhanced version of the multiclaim dataset @xcite , specifically tailored to meet these multilingual needs. our system uses a zero-shot approach based on pre-trained text embedding models (tems). we selected three tems according to preliminary experiments and used their combination to further improve the results and make the system more robust. all models and combinations were evaluated on the development data. the final approach employs the best model or combination for each language. we use all available text (incl. ocr) as input to have a maximal context. in some cases, though, we encountered model limits regarding maximum input size. the input was truncated to fit the model's tokenizer in such a case. based on preliminary experiments where multilingual models used with original texts achieved worse results, we use only english translations in the final approach. our contributions are as follows: 1) we compare several embedding models and show that simple zero-shot embeddings are effective for the task. 2) we demonstrate that larger models consistently outperform smaller ones. 3) we propose combining different models for different languages, which improves overall performance. the following section highlights the most essential facts about the task and related work focuses on tems. then, we present our approach, experimental setup, and results, including the error analysis. the final section concludes the paper. ## background the basis for this semeval-2025 task was presented in the paper multilingual previously fact-checked claim retrieval @xcite . the motivation for this task is to reliably identify previously fact-checked claims for various social media posts across multiple languages. furthermore, this task addresses the challenge of both multilingual and cross-lingual settings, providing valuable support to fact-checkers and researchers in combating the global spread of misinformation. the dataset for evaluation has been derived from the existing multiclaim dataset @xcite with some specific modifications. the dataset contains tens of thousands of multilingual social media posts, matched with over 200,000 fact-checks across nearly 40 different languages. in addition to the text of the posts, ocr output is also available (when a post includes an image), including language identification (if the image contains multilingual texts). moreover, there are english translations for all texts and also a verdict that shows a label of the post (e.g. false or partly false information, altered photo, etc.) and a list of timestamps. ## the proposed approach the system operates in a zero-shot setting, leveraging multiple tems to obtain sentence representations. specifically, we employ nvidia nv-embed-v2 (nv-embed) @xcite , base multilingual gte (mgte) @xcite , large english gte @xcite , gpt text-embedding-3-large (openai, 2025), and mistral mistral-embed (ai, 2025). these models are based on the transformer architecture @xcite , which processes an input sentence @xmath0, where h i is a hidden feature representation for a corresponding token x i . several techniques exist to obtain a vector representation of a full sentence. the gte models prepend a special [cls] token at the beginning of the sequence, which serves as a global representation of the sentence. nv-embed utilizes a latent attention layer to generate the final sentencelevel vector. another common approach is a mean pooling, which averages the token-level vectors to produce a single representation. the nv-embed model requires a prompt to be prefixed to input queries (in our case, posts). we use the following prompt: "given a post, retrieve claims that verify the post". for each input post, we concatenate the original text with an ocr-extracted text. we use english translations for all models except for mgte, where we use texts in their original language. we feed the concatenated text into the models without additional preprocessing. similarly, we feed the fact-checks into a tem concatenating the title and the claim. as a result, we obtain a vector x @xmath1, where n is the number of candidate fact-checks (claims) and e is the embedding dimension based on the utilized tem. the goal is to find the 10 closest rows (vectors) in the matrix based on the cosine similarity. ## experiments the task organizers have chosen success-at-k (s@k) as a main evaluation metric. the metric expresses a percentage of pairs when at least one of the desired fact-checks ends up in the top k, where @xmath2. the s@k for each particular language is computed separately, and the mean value represents the multilingual result. ## conclusion this paper described our approach for the semeval-2025 shared task 7 multilingual and cross-lingual fact-checked claim retrieval. we adopted a zeroshot approach using large language models like nvidia nv-embed-v2, gpt text-embedding-3large, and mistral. this approach allows seamless integration of new languages without retraining, as demonstrated when polish and turkish were added to the test set. by leveraging the embeddings and measuring cosine similarity, we identified the most relevant claims for each post. our approach ranked 7th out of 28 in the monolingual subtask and 9th out of 29 in the crosslingual subtask. error analysis showed that all three models effectively placed the most relevant claims at the top of the ranked lists. for some languages, combining models improved performance, and our final submission reflected this. among the models, nv-embed proved the most effective, keeping the number of missed pairs below 25% in the cross-lingual setting.