OpenRaiser/Intern-Atlas huggingface dataset:
Flow-guided Direct Preference Optimization for Knowledge Graph Reasoning with Trees
Authors: Tiesunlong Shen, Rui Mao, Jin Wang, Xuejie Zhang, Erik Cambria
Venue: SIGIR
Year: 2025

Recent advancements in knowledge graph question answering (KGQA) have shown promise, yet existing methods often fail to align with human reasoning patterns that involve continuous reflection and refinement. This paper proposes FD-PORT (flow-guided direct preference optimization for knowledge graph reasoning with trees), a novel approach that combines Monte Carlo Tree Search (MCTS) with flow-guided direct preference optimization (FDPO) for KGQA tasks. MCTS simulates human-like reasoning by systematically exploring multiple inference paths in knowledge graphs, while FDPO transforms the search feedback into fine-grained training signals through flow balance conditions. Unlike traditional methods focusing on end-to-end training or sequence-level preferences, FD-PORT establishes flow consistency between any states along the reasoning chain, enabling robust multi-hop reasoning that adapts to local decisions and long-range dependencies. Experimental results on three benchmark datasets demonstrate that FD-PORT significantly outperforms state-of-the-art methods, achieving up to 50.6% improvements over GPT-4 on complex multi-hop reasoning tasks with a smaller open-source language model. The framework is advanced in maintaining diverse reasoning paths while ensuring answer quality, closely mirroring human problem-solving strategies.
Exploring ℓ0 Sparsification for Inference-free Sparse Retrievers
Authors: Xinjie Shen, Zhichao Geng, Yang Yang
Venue: SIGIR
Year: 2025

With increasing demands for efficiency, information retrieval has developed a branch of sparse retrieval, further advancing towards inference-free retrieval where the documents are encoded during indexing time and there is no model-inference for queries. Existing sparse retrieval models rely on FLOPS regularization for sparsification, while this mechanism was originally designed for Siamese encoders, it is considered to be suboptimal in inference-free scenarios which is asymmetric. Previous attempts to adapt FLOPS for inference-free scenarios have been limited to rule-based methods, leaving the potential of sparsification approaches for inference-free retrieval models largely unexplored. In this paper, we explore ℓ0 inspired sparsification manner for inference-free retrievers. Through comprehensive out-of-domain evaluation on the BEIR benchmark, our method achieves state-of-the-art performance among inference-free sparse retrieval models and is comparable to leading Siamese sparse retrieval models. Furthermore, we provide insights into the trade-off between retrieval effectiveness and computational efficiency, demonstrating practical value for real-world applications.
Alleviating LLM-based Generative Retrieval Hallucination in Alipay Search
Authors: Yedan Shen, Kaixin Wu, Yuechen Ding, Jingyuan Wen, Hong Liu, Mingjie Zhong, Zhouhan Lin, Jia Xu, Linjian Mo
Venue: SIGIR
Year: 2025

Generative retrieval (GR) has revolutionized document retrieval with the advent of large language models (LLMs), and LLM-based GR is gradually being adopted by the industry. Despite its remarkable advantages and potential, LLM-based GR suffers from hallucination and generates documents that are irrelevant to the query in some instances, severely challenging its credibility in practical applications. We thereby propose an optimized GR framework designed to alleviate retrieval hallucination, which integrates knowledge distillation reasoning in model training and incorporate decision agent to further improve retrieval precision. Specifically, we employ LLMs to assess and reason GR retrieved query-document (q-d) pairs, and then distill the reasoning data as transferred knowledge to the GR model. Moreover, we utilize a decision agent as post-processing to extend the GR retrieved documents through retrieval model and select the most relevant ones from multi perspectives as the final generative retrieval result. Extensive offline experiments on real-world datasets and online A/B tests on Fund Search and Insurance Search in Alipay demonstrate our framework's superiority and effectiveness in improving search quality and conversion gains.
Retrieval-Augmented Image Captioning and Generation with Entity Concepts Enhancement for Baidu Multimodal Advertising
Authors: Lei Shen, Kang Zhao, Zhipeng Jin, Wen Tao, Yi Yang, Cong Han, Shuanglong Li, Zhongmin Cai, Lin Liu
Venue: SIGIR
Year: 2025

Recent advancements in generative artificial intelligence are driving a significant transformation in information retrieval and content generation, creating substantial opportunities for online advertising. Text-to-image generation technology has become increasingly prevalent in advertising content production, demonstrating promising performance improvements in terms of semantic relevance and visual appeal. However, existing models often suffer from inadequate representation of entity concepts, such as prominent product brands and recognizable landmarks. This inherent limitation subsequently leads to notable deficiencies in brand tonality, industry-specific relevance, and market adaptability of the generated advertising content. To address this challenge, we propose a multimodal ad content generation framework specifically engineered for online advertising system, particularly focused on resolving the deficiency in entity concepts. Our framework is comprised of two phases: first, an image captioning module with entity-aware learning based on multimodal large language model, leveraging retrieval-augmented techniques to incorporate entity concepts into image descriptions; second, a text-to-image diffusion model refined on image-text pairs enriched with entity concepts to facilitate entity-grounded image generation. Extensive experiments validate the effectiveness of our framework, demonstrating superior performance in both image captioning and image generation compared to existing methods, particularly in the accuracy of depiction of relevant entities in advertising images. Moreover, the deployment of the framework in the system primary traffic of Baidu Search Ads, has brought significant enhancements to advertisement revenue for both advertisers and the platform.
Progressive Refinement of E-commerce Search Ranking Based on Short-Term Activities of the Buyer
Authors: Taoran Sheng, Sathappan Muthiah, Atiq Islam, Jinming Feng
Venue: SIGIR
Year: 2025

In e-commerce shopping, aligning search results with a buyer's immediate needs and preferences presents a significant challenge, particularly in adapting search results throughout the buyer's shopping journey as they move from the initial stages of browsing to making a purchase decision or shift from one intent to another. This study presents a systematic approach to adapting e-commerce search results based on the current context. We start with basic methods and incrementally incorporate more contextual information and state-of-the-art techniques to improve the search outcomes. By applying this evolving contextual framework to items displayed on the search engine results page (SERP), we progressively align search outcomes more closely with the buyer's interests and current search intentions. Our findings demonstrate that this incremental enhancement, from simple heuristic autoregressive features to advanced sequence models, significantly improves ranker performance. The integration of contextual techniques enhances the performance of our production ranker, leading to improved search results in both offline and online A/B testing in terms of Mean Reciprocal Rank (MRR). Overall, the paper details iterative methodologies and their substantial contributions to search result contextualization on e-commerce platforms.
Retrieval Augmented Generation with Collaborative Filtering for Personalized Text Generation
Authors: Teng Shi, Jun Xu, Xiao Zhang, Xiaoxue Zang, Kai Zheng, Yang Song, Han Li
Venue: SIGIR
Year: 2025

Recently, the personalization of Large Language Models (LLMs) to generate content that aligns with individual user preferences has garnered widespread attention. Personalized Retrieval-Augmented Generation (RAG), which retrieves relevant documents from the user's history to reflect their preferences and enhance LLM generation, is one commonly used approach for personalization. However, existing personalized RAG methods do not consider that the histories of similar users can also assist in personalized generation for the current user, meaning that collaborative information between users can also benefit personalized generation. Inspired by the application of collaborative filtering in recommender systems, we propose a method called CFRAG, which adapts Collaborative Filtering to RAG for personalized text generation. However, this presents two challenges: (1) how to incorporate collaborative information without explicit user similarity labels? (2) how to retrieve documents that support personalized LLM generation? For Challenge 1, we use contrastive learning to train user embeddings to retrieve similar users and introduce collaborative information. For Challenge 2, we design a personalized retriever and reranker to retrieve the top-k documents from these users' histories. We take into account the user's preference during retrieval and reranking. Then we leverage feedback from the LLM to fine-tune the personalized retriever and reranker, enabling them to retrieve documents that meet the personalized generation needs of the LLM. Experimental results on the Language Model Personalization (LaMP) benchmark validate the effectiveness of CFRAG. Further analysis confirms the importance of incorporating collaborative information.
LLM4Eval: Large Language Model for Evaluation in IR
Authors: Clemencia Siro, Hossein A. Rahmani, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
Venue: SIGIR
Year: 2025

Large language models (LLMs) have demonstrated increasing task-solving abilities not present in smaller models. Utilizing the capabilities and responsibilities of LLMs for automated evaluation (LLM4Eval) has recently attracted considerable attention in multiple research communities. Building on the success of previous workshops, which established foundations in automated judgments and RAG evaluation, this third iteration aims to address emerging challenges as IR systems become increasingly personalized and interactive. The main goal of the third LLM4Eval workshop is to bring together researchers from industry and academia to explore three critical areas: the evaluation of personalized IR systems while maintaining fairness, the boundaries between automated and human assessment in subjective scenarios, and evaluation methodologies for systems that combine multiple IR paradigms (search, recommendations, and dialogue). By examining these challenges, we seek to understand how evaluation approaches can evolve to match the sophistication of modern IR applications. The format of the workshop is interactive, including roundtable discussion sessions, fostering dialogue about the future of IR evaluation while avoiding one-sided discussions. This is the third iteration of the workshop series, following successful events at SIGIR 2024 and WSDM 2025, with the first iteration attracting over 50 participants.
From Query to Conscience: The Importance of Information Retrieval in Empowering Socially Responsible Consumerism
Authors: Frans van der Sluis, Leif Azzopardi, Florian Meier
Venue: SIGIR
Year: 2025

Millions of consumers search for products online each day, aiming to find items that meet their needs at an acceptable price. While price and quality are major factors in purchasing decisions, ethical considerations increasingly influence consumer behavior -giving rise to the socially responsible consumer. Insights from a recent survey of over 600 consumers reveal that many barriers to ethical shopping stem from information-seeking challenges, often leading to decisions made under uncertainty. These challenges contribute to the intention-behaviour gap, where consumers' desire to make ethical choices is undermined by limited or inaccessible information and inefficacy of search systems in supporting responsible decision-making. In this perspectives paper, we argue that the field of Information Retrieval (IR) has a critical role to play by empowering consumers to make more informed and more responsible choices. We present three interrelated perspectives: (1) reframing ethical consumption as an information extraction problem aimed at reducing information asymmetries; (2) redefining product search as a complex task requiring interfaces that lower the cost and burden of responsible search; and (3) reimagining search as a process of knowledge calibration that helps consumers bridge gaps in awareness when making purchasing decisions. Taken together, these perspectives outline a path from query to conscience - one where IR systems help transform everyday product searches into opportunities for more ethical and informed choices. We advocate for the development of new and novel IR systems and interfaces that address the intricacies of socially responsible consumerism, and call on the IR community to build technologies that make ethical decisions more informed, convenient, and aligned with economic realities.
Extending MovieLens-32M to Provide New Evaluation Objectives
Authors: Mark D. Smucker, Houmaan Chamani
Venue: SIGIR
Year: 2025

Offline evaluation of recommender systems has traditionally treated the problem as a machine learning problem. In the classic case of recommending movies, where the user has provided explicit ratings of which movies they like and don't like, each user's ratings are split into test and train sets, and the evaluation task becomes to predict the held out test data using the training data. This machine learning style of evaluation makes the objective to recommend the movies that a user has watched and rated highly, which is not the same task as helping the user find movies that they would enjoy if they watched them. This mismatch in objective between evaluation and task is a compromise to avoid the cost of asking a user to evaluate recommendations by watching each movie. As a resource available for download, we offer an extension to the MovieLens-32M dataset that provides for new evaluation objectives. Our primary objective is to predict the movies that a user would be interested in watching, i.e. predict their watchlist. To construct this extension, we recruited MovieLens users, collected their profiles, made recommendations with a diverse set of algorithms, pooled the recommendations, and had the users assess the pools. This paper demonstrates the feasibility of using pooling to construct a test collection for recommender systems. Notably, we found that the traditional machine learning style of evaluation ranks the Popular algorithm, which recommends movies based on total number of ratings in the system, in the middle of the twenty-two recommendation runs we used to build the pools. In contrast, when we rank the runs by users' interest in watching movies, we find that recommending popular movies as a recommendation algorithm becomes one of the worst performing runs. It appears that by asking users to assess their personal recommendations, we can alleviate the issue of popularity bias in the evaluation of top-n recommendation.
Reinforcement Learning for Effective Few-Shot Ranking
Authors: Shiva Soleimany, Sajad Ebrahimi, Shirin Seyedsalehi, Fattane Zarrinkalam, Ebrahim Bagheri
Venue: SIGIR
Year: 2025

Neural rankers have achieved strong retrieval effectiveness but require large amounts of labeled data, limiting their applicability in few-shot settings. In this paper, we address the sample inefficiency of neural ranking methods by introducing a Reinforcement Learning (RL)-based re-ranking model that achieves high effectiveness with minimal training data. Built on a Deep Q-learning Network (DQN) framework, our approach is designed for few-shot settings, maximizing sample efficiency to ensure robust generalization from limited interactions. Extensive experiments show that our model significantly outperforms data-intensive methods and existing few-shot baselines, demonstrating RL's potential to enhance IR capabilities in few-shot scenarios.
Rating-Aware Homogeneous Review Graphs and User Likes/Dislikes Differentiation for Effective Recommendations
Authors: Jiwon Son, Hyunjoon Kim, Sang-Wook Kim
Venue: SIGIR
Year: 2025

The goal of Review-Based Recommendation System (RBRS) is to effectively learn the representations of users and items by utilizing review texts in addition to user-item interactions. From user-item interaction graphs widely employed in recommendation systems, recent RBRS methods using graph neural networks (GNNs) obtain the representations by associating each edge between a user and an item with the review information of the user for that item. However, these GNN-based RBRS methods present two main issues: (1) by con- verting each review text into the weight, i.e., single value, of a edge between a user node and an item node, they lose the rich informa- tion about users and items inherent in the review; and (2) by creating only a single general representation for each user, they cannot repre- sent the individual effects of users' likes and dislikes on their ratings for items they have interacted with. To address these problems, we propose a novel GNN-based RBRS, named LETTER, utilizing homo- geneous graphs, i.e., user-user graphs and an item-item graph, to learn general representations of users and items along with users' like and dislike representations. LETTER can learn user and item representations without losing review information by utilizing the proposed homogeneous graphs. Furthermore, LETTER explicitly designs the influence of users' like and dislike representations on their ratings to perform accurate rating predictions. Through ex- periments on six datasets, we verify that the proposed LETTER out- performs nine state-of-the-art RBRSs by up to 23.1%. Our source code is available at https://github.com/Bigdasgit/LETTER.
Bridging Time Gaps: Temporal Logic Relations for Enhancing Temporal Reasoning in Large Language Models
Authors: Xintong Song, Bin Liang, Yang Sun, Chenhua Zhang, Bingbing Wang, Ruifeng Xu
Venue: SIGIR
Year: 2025

The understanding and cognition of time are the basis for large language models to understand the world. Although large language models (LLMs) have demonstrated strong capabilities in multiple reasoning tasks, they still have significant deficiencies in temporal reasoning, mainly due to the diversity of temporal expressions and the lack of temporal logic reasoning capabilities. In this study, we propose a novel Temporal Chain of Thought framework(TempCoT) to improve the performance of LLM in temporal reasoning tasks through a three-stage reasoning strategy. First, TempCoT explicitly extracts time constraints to ensure the accuracy of time references during reasoning. Second, a semantic retrieval mechanism is introduced to dynamically obtain key temporal facts to enhance the integrity and reliability of information. Finally, an explicit temporal logic reasoning module is constructed based on point algebra to improve the consistency and interpretability of reasoning. Experimental results show that TempCoT significantly improves the temporal reasoning performance of five different LLMs and shows stronger robustness on complex temporal tasks.
LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction
Authors: Xin Song, Xiaochen Li, Jinxin Hu, Hong Wen, Zulong Chen, Yu Zhang, Xiaoyi Zeng, Jing Zhang
Venue: SIGIR
Year: 2025

With the rapid growth of user historical behavior data, user interest modeling has become a prominent aspect in Click-Through Rate (CTR) prediction, focusing on learning user intent representations. However, this complexity poses computational challenges, requiring a balance between model performance and acceptable response times for online services. Traditional methods often utilize filtering techniques. These techniques can lead to the loss of significant information by prioritizing top K items based on item attributes or employing low-precision attention mechanisms. In this study, we introduce LREA, a novel attention mechanism that overcomes the limitations of existing approaches while ensuring computational efficiency. LREA leverages low-rank matrix decomposition to optimize runtime performance and incorporates a specially designed loss function to maintain attention capabilities while preserving information integrity. During the inference phase, matrix absorption and pre-storage strategies are employed to effectively meet runtime constraints. The results of extensive offline and online experiments demonstrate that our method outperforms state-of-the-art approaches.
Dual-perspective Data Augmentation and Curriculum Learning Framework for Low-resource Complex Named Entity Recognition
Authors: Mengxiao Song, Tianyun Liu, Wenyuan Zhang, Quangang Li, Tingwen Liu
Venue: SIGIR
Year: 2025

Low-resource complex named entity recognition focuses on identifying complex entities such as creative work, product name and so on, in scenarios where annotated training data is limited. Recent advanced works deal with this task through data augmentation and make substantial progress. However, existing methods ignore the influence of different types or levels of augmented data on model optimization in different learning stages. To address it, we propose a dual-perspective data augmentation and curriculum learning framework. Specifically, we first employ the large language model (LLM) to construct two kinds of augmented datasets from context-perspective and entity-perspective, respectively. Then, we present a multi-stage curriculum learning strategy including a novel adaptive curriculum arrangement algorithm to automatically select the most suitable kind of augmented set to optimize the target model at each training epoch, thus using the augmented data more effectively and controllably. Experimental results on the public benchmark across various low-resource settings show that our framework outperforms previous works.
The Magnitude of Truth: On Using Magnitude Estimation for Truthfulness Assessment
Authors: Michael Soprano, Denis Eduard Tapu, David La Barbera, Kevin Roitero, Stefano Mizzaro
Venue: SIGIR
Year: 2025

Assessing the truthfulness of information is a critical task in fact-checking, and is typically performed using binary or coarse ordinal scales (2-6 levels), though fine-grained scales (e.g., 100 levels) have also been explored. Magnitude Estimation (ME) takes this approach further by allowing assessors to assign any value in the range (0, + ∞). However, it introduces challenges, including the need for aggregation of assessments from individuals with different interpretations of the scale. Despite these, its successful applications in other domains suggest its potential suitability for truthfulness assessment. We conduct a crowdsourcing study by collecting assessments on claims sourced from the PolitiFact fact-checking organization using ME. To the best of our knowledge, this is the first systematic investigation of ME in the context of truthfulness assessment. Our results show that while aggregation methods significantly impact assessment quality, optimal aggregation strategies yield accuracy and reliability comparable to traditional scales. More importantly, ME allows capturing subtle differences in truthfulness, offering richer insights than conventional coarse-grained scales.