A lex UNLP - FMT at C limate C heck Shared Task: Hybrid Retrieval with Adaptive Similarity Graph-based Reranking for Climate-related Social Media Claims Fact Checking (2025)

Abstract: In this paper, we describe our work done in the ClimateCheck shared task at the Scholarly document processing (SDP) workshop, ACL 2025. We focused on subtask 1: Abstracts Retrieval. The task involved retrieving relevant paper abstracts from a large corpus to verify claims made on social media about climate change. We explored various retrieval and ranking techniques, including fine-tuning transformer-based dense retrievers, sparse retrieval methods, and reranking using cross-encoder models. Our final and best-performing system utilizes a hybrid retrieval approach combining BM25 sparse retrieval and a fine-tuned Stella model for dense retrieval, followed by an MSMARCO trained minilm cross-encoder model for ranking. We adapt an iterative graph-based re-ranking approach leveraging a document similarity graph built for the document corpus to dynamically update candidate pool for reranking. This system achieved a score of 0.415 on the final test set for subtask 1, securing 3rd place in the final leader board.

## introduction misinformation spreading on social media poses a significant threat to public understanding of scientific issues, particularly in domains such as climate change, where accurate information is needed to raise awareness and create evidence-based policies. social media platforms are often the first point of exposure to climate-related content for the general public, making it easy for misleading claims and information to spread. therefore, there is a need for automated fact-checking systems that can assess the veracity of such claims in real time. automated evidence-based fact-checking remains a challenging task @xcite . highly effective retrieval module that can retrieve relevant evidence to support or refute a given claim is a necessary component of the evidence-based fact-checking system @xcite . this paper presents our approach for subtask 1 of the climatecheck shared task @xcite , held at the scholarly document processing (sdp) workshop at acl 2025. the subtask focuses on retrieving relevant scientific abstracts from a large corpus in response to climate-related claims made on social media. we experiment with dense and sparse retrieval models and employ a retrieval-reranking pipeline. we fine-tune models using supervised contrastive learning and evaluate the effectiveness of hybrid retrieval pipelines that combine sparse and dense approaches. we adapt a graph-based reranking approach inspired by prior work on corpus graph expansion @xcite , where the reranking pool is iteratively enriched using neighbors of top-ranked documents. ## related work information retrieval (ir) pipelines generally rely on sparse or dense retrieval techniques. sparse retrieval sparse retrieval methods, such as bm25 @xcite , represent queries and documents as high-dimensional sparse vectors based on term frequency-inverse document frequency statistics (tf-idf). while effective in capturing lexical similarity, these models often struggle to capture semantic similarity. dense retrieval dense retrieval models @xcite @xcite address the problem of semantic similarity faced by sparse retrieval models. they embed both queries and documents to obtain dense vector representations that allow similarity-based search through vector similarity. adaptive reranking adaptive reranking techniques that utilize similarity graphs @xcite have been developed to overcome the limitations of standard retrieval-reranking pipelines, where the reranking performance is limited by the set initially retrieved by the retriever @xcite . in the adaptive reranking approach, a similarity graph is used to retrieve documents related to the top-ranked ones, enabling richer reranking candidates. ## data the shared task provided a training set, a test set, and a document corpus @xcite . the training set included 1,144 claim-abstract pairs labeled as supports, refutes, or not enough information; the label distribution is shown in since the shared task initially provided only a training set, we created our own validation set by randomly sampling 50 unique claims and their associated data points from the original training set. the remaining samples formed our training set. distributions for both sets are shown in ## methodology in this section, we outline our approach and the final submitted system. ## experiments and results evaluation on the validation and test sets was performed using the official shared task metrics: recall@k (r@k for @xmath0, 5, 10), b-pref, and the overall subtaski-score, defined as the average of the other metrics. ## conclusion in this paper, we presented a hybrid retrieval system with adaptive reranking for evidence retrieval for climate-related social media claims, developed for subtask 1 of the climatecheck shared task. our system combined bm25-based sparse retrieval with a fine-tuned dense retriever, followed by a graph-based adaptive reranking approach utilizing a document similarity graph. we demonstrated that hybrid retrieval paired with iterative reranking significantly improved retrieval effectiveness, achieving third place in the final leaderboard. our findings emphasize the importance of combining hybrid retrieval with adaptive reranking to enhance the performance of scientific evidence retrieval systems. the use of graph-based expansion enabled the discovery of relevant abstracts that were missed by standard top-k methods.