NCL - AR at S em E val-2025 Task 7: A Sieve Filtering Approach to Refute the Misinformation within Harmful Social Media Posts (2025)

Abstract: In this paper, we propose a sieve filtering-based approach that can retrieve facts to invalidate claims made in social media posts. The fact filters are initially coarse-grained, based on the original language of the social media posts, and end with fine-grained filters based on the exact time frame in which the posts were uploaded online. This streamlined approach achieved a 0.883 retrieval success rate in the monolingual task while maintaining a scalable efficiency level of processing a social media post per 0.07 seconds.

## introduction societally, we have been transitioning into a world of misinformation, disinformation, and fake news @xcite . however, this transition has accelerated dramatically due to the recent advancement in generative ai, making it easier and more accessible to create highly believable and persuasive text, images, and videos @xcite . the ability to post online as a form of expression, collaboration, and connection worldwide is becoming increasingly tainted by the spread of false and misleading information. this issue is becoming increasingly prevalent as widespread misinterpretations of these posts are fuelling polarisation, inciting hatred, and causing harm to the population @xcite . within this problem arises an essential need for fact-checking systems to combat this new threat. these systems must be highly accurate to counteract the false information and avoid amplifying the current issue. in addition, they must be efficient and multilingual, as around 5.22 billion social media users worldwide post and share content daily in various languages. the systems must identify and mitigate misinformation in real-time before it spreads @xcite . in the context of previously fact-checked claim retrieval (pfcr) @xcite , task 2 of clef 2022 focused on competitors determining a list of top-n fact-checked claims to corresponding tweets or political dialogues @xcite . one of the published methods from the task, which achieved the highest accuracy, used a 3-step pipeline of pre-processing the tweets for clarity, retrieving the relevant claims using the bm25 model, and generatively re-ranking the claims using gpt-neo-1.3b @xcite . another notable approach used the sentence transformer models, all-minilm-l6-v2 and all-mpnet-base-v2, alongside a support machine vector model to create an ensemble classifier to calculate the similarity score between tweets and fact-checked claims @xcite . task 7 of semeval-2025 aimed to address the challenge of misinformation @xcite . participants were tasked with developing a fact retrieval system capable of providing relevant facts to contextualise or disprove social media posts. in this paper, we propose a sieve filtering-based approach that can retrieve facts to invalidate claims made in social media posts. the fact filters are initially coarse-grained, based on the original language of the social media posts, and end with fine-grained filters based on the exact time frame in which the posts were uploaded online. this streamlined approach achieved a 0.883 retrieval success rate in the monolingual task while maintaining a scalable efficiency level of processing a social media post per 0.07 seconds. ## dataset the organisers used a modified version of the mul-ticlaim dataset, enhanced especially for the task @xcite . the authors provided a development dataset consisting of a post csv p and fact-check csv f , containing 24,431 posts and 153,743 fact-checks. in the testing dataset, this changed, with the post csv changing to 8,276 unseen posts and the fact-check cvs expanding with an additional 118,704 unseen fact-checks, bringing the total number of fact-checks to 272,447. each social media post within the dataset was flagged by a professional fact-checker from the social media sites facebook, instagram and twitter. each post entry p includes the post id p id , post unix timestamp p time , social media platform p plat , post verdict p ver , post content p context and ocr-extracted text p ocr , which both contain the text in the original language, english translation, and predicted language country code with the accuracy rating. ## methodology to tackle the task, we propose a simplistic sieve approach, as illustrated in figure 1 , to filter the f based on the post's features (p time , p context , p ocr ) until the top 10 most relevant f to the p are identified. ## evaluation setup the evaluation of semeval-2025 task 7 was split into two tracks: monolingual and crosslingual. the claim and the retrieved fact-check were in the same language in the monolingual track. whereas the crosslingual track meant the claim and the retrieved fact-check may be in different languages from the 27 available in the test dataset, which provided an extra dimension of complexity to the challenge. ## conclusion in this paper, we presented our approach in the semeval-2025 task 7, which addressed the ongoing challenge of fact-checking social media posts to debunk the misinformation contained within. the task has an additional difficulty component, as the facts and posts are from one of 27 different languages, reflecting the global importance of this challenge. our sieve filtering approach used the key features, such as original language, semantic content, and time frame, from each social media post to retrieve the 10 most relevant fact-checks. our approach achieved a 0.883 success rate when retrieving fact-post parings in the monolingual task without sacrificing the high-efficiency levels of 0.07 seconds to disprove each post factually. while our approach does not match the top approaches in the leaderboard, it provides a strong baseline for future improvements. one future improvement for the approach would be to use the other features within each fact-check, such as the article url, to provide more context and potentially improve retrieval accuracy.