CAISA at S em E val-2025 Task 7: Multilingual and Cross-lingual Fact-Checked Claim Retrieval (2025)

Abstract: We leveraged LLaMA, utilizing its ability to evaluate the relevance of retrieved claims within a retrieval-based fact-checking framework. This approach aimed to explore the impact of large language models (LLMs) on retrieval tasks and assess their effectiveness in enhancing fact-checking accuracy. Additionally, we integrated Jina embeddings v2 and the MPNet multilingual sentence transformer to filter and rank a set of 500 candidate claims. These refined claims were then used as input for LLaMA, ensuring that only the most contextually relevant ones were assessed.

## introduction the rapid proliferation of misinformation in recent years has raised significant concerns across governments, organizations, and individuals. false information spreads rapidly through online platforms, influencing public opinion and decisionmaking @xcite . to combat this, numerous fact-checking organizations, such as factcheck.org and snopes, have emerged to manually verify claims. however, manual fact-checking is labor-intensive and struggles to keep pace with the high volume of misinformation circulating online @xcite . misinformation is a global issue, requiring factchecking solutions that extend beyond monolingual settings. to address this challenge, researchers have developed automated fact-checking systems, including verified claim retrieval, where the goal is to find previously fact-checked claims relevant to an input claim @xcite . this task is particularly crucial as many false claims resurface in different forms across time and languages. while prior work has largely focused on monolingual claim retrieval, real-world misinformation often transcends language barriers, making multilingual claim retrieval a pressing issue. as illustrated in figure 1 , we tackle multilingual verified claim retrieval by leveraging llama @xcite , a large language model (llm) capable of understanding and retrieving fact-checked claims across different languages. to enhance retrieval precision, we incorporate jina embeddings v2 @xcite and the mp-net multilingual sentence transformer @xcite , filtering and ranking candidate claims before passing them to llama. by extending verified claim retrieval to a multilingual setting, we contribute to developing cross-lingual fact-checking systems that help mitigate misinformation more effectively. ## related work multilingual claim retrieval. @xcite introduced multiclaim, the largest dataset for multilingual claim retrieval, featuring 28k posts (27 languages) and 206k fact-checks (39 languages). their study shows that supervised fine-tuning improves retrieval performance over unsupervised methods. monolingual claim matching. previous work has explored retrieval-based ranking for claim matching, using bert and bm25 to rank checkworthy claims @xcite . however, their dataset focused only on political claims in monolingual settings. cross-lingual claim matching. efforts to match social media claims with fact-checks across languages have utilized xlm-roberta, bm25, and labse @xcite . while effective monolingually, cross-lingual performance remains a challenge. multimodal fact-checking. ragar: your falsehood radar explores retrieval-augmented generation (rag) for political claims, integrating textual and image inputs. while corag and torag reasoning techniques improve verification, reliance on gpt-4v (closed-source) limits adaptability @xcite . end-to-end multimodal verification. an sbert-bart framework combines evidence retrieval, claim verification, and explanation generation @xcite . however, sbert struggles with cross-modal reasoning, affecting its ability to integrate text and visual data effectively. ## conclusion in conclusion, our approach to retrieving previously fact-checked claims using the llama model across multiple languages has shown promising results, achieving a success@10 value of 55.7 percent. this demonstrates that llama can effectively assist in automating fact-checking tasks, making the process more efficient and scalable. llama performs well in multilingual claim retrieval because it has a strong understanding of context and semantics across different languages. as a large language model (llm), it is trained on diverse multilingual data, allowing it to recognize paraphrased or reworded claims. its attentionbased architecture helps it detect long-range dependencies, making it effective in finding related fact-checked claims even when the wording differs. additionally, using llama with task description prompt on claim verification tasks improves its accuracy by aligning its understanding with realworld fact-checking datasets. while the current success rate is encouraging, further refinements and optimizations are needed to improve the accuracy and reliability of claim retrieval across diverse languages. despite its strengths, llama has some limitations. it sometimes overgeneralizes, meaning it might incorrectly match a claim with a similar but factually incorrect statement. additionally, since llama is not a dedicated fact-checking model, it may hallucinate relationships between claims that do not exist.