OpenRaiser/Intern-Atlas huggingface dataset:
Improving LLM-powered Recommendations with Personalized Information
Authors: Jiahao Liu, Xueshuo Yan, Dongsheng Li, Guangping Zhang, Hansu Gu, Peng Zhang, Tun Lu, Li Shang, Ning Gu
Venue: SIGIR
Year: 2025

Due to the lack of explicit reasoning modeling, existing LLM-powered recommendations fail to leverage LLMs' reasoning capabilities effectively. In this paper, we propose a pipeline called CoT-Rec, which integrates two key Chain-of-Thought (CoT) processes-user preference analysis and item perception analysis-into LLM-powered recommendations, thereby enhancing the utilization of LLMs' reasoning abilities. CoT-Rec consists of two stages: (1) personalized information extraction, where user preferences and item perception are extracted, and (2) personalized information utilization, where this information is incorporated into the LLM-powered recommendation process. Experimental results demonstrate that CoT-Rec shows potential for improving LLM-powered recommendations. The implementation is publicly available at https://github.com/jhliu0807/CoT-Rec.
Multi-objective Aligned Bidword Generation Model for E-commerce Search Advertising
Authors: Zhenhui Liu, Chunyuan Yuan, Ming Pang, Zheng Fang, Li Yuan, Xue Jiang, Changping Peng, Zhangang Lin, Zheng Luo, Jingping Shao
Venue: SIGIR
Year: 2025

The retrieval system is a crucial module in e-commerce search advertising that matches user queries with ads. The diverse expressions of users often produce massive tail queries that cannot match merchant bidwords, leading to poor retrieval efficiency. Existing methods, such as query log mining and vector matching, fail to optimize relevance, authenticity, and ad revenue of the rewrite. In this paper, we propose a novel Multi-objective aligned Bidword Generation Model (MoBGM), which includes a discriminator, generator, and preference alignment module. To simultaneously improve the relevance and authenticity of the query and rewrite and maximize the platform revenue, we design a discriminator to predict these scores for each query and rewrite pair. Using the feedback signal of the discriminator, we train a multi-objective aligned bidword generator to maximize the combined effect of the three objectives. Extensive offline and online experiments show that our proposed algorithm significantly outperforms the state of the art. After deployment, the algorithm created huge commercial value for the platform, verifying its feasibility and robustness.
WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers
Authors: Mugeng Liu, Siqi Zhong, Qi Yang, Yudong Han, Xuanzhe Liu, Yun Ma
Venue: SIGIR
Year: 2025

Approximate nearest neighbor search (ANNS) has become vital to modern AI infrastructure, particularly in retrieval-augmented generation (RAG) applications. Numerous in-browser ANNS engines have emerged to seamlessly integrate with popular LLM-based web applications, while addressing privacy protection and challenges of heterogeneous device deployments. However, web browsers present unique challenges for ANNS, including computational limitations, external storage access issues, and memory utilization constraints, which state-of-the-art (SOTA) solutions fail to address comprehensively. We propose WebANNS, a novel ANNS engine specifically designed for web browsers. WebANNS leverages WebAssembly to overcome computational bottlenecks, designs a lazy loading strategy to optimize data retrieval from external storage, and applies a heuristic approach to reduce memory usage. Experiments show that WebANNS is fast and memory efficient, achieving up to 743.8X improvement in 99th percentile query latency over the SOTA engine, while reducing memory usage by up to 39%. Note that WebANNS decreases query time from 10 seconds to the 10-millisecond range in browsers, making in-browser ANNS practical with user-acceptable latency.
Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information
Authors: Kehan Long, Shasha Li, Chen Xu, Jintao Tang, Ting Wang
Venue: SIGIR
Year: 2025

Recent advancements have successfully harnessed the power of Large Language Models (LLMs) for zero-shot document ranking, exploring a variety of prompting strategies. Comparative approaches like pairwise and listwise achieve high effectiveness but are computationally intensive and thus less practical for larger-scale applications. Scoring-based pointwise approaches exhibit superior efficiency by independently and simultaneously generating the relevance scores for each candidate document. However, this independence ignores critical comparative insights between documents, resulting in inconsistent scoring and suboptimal performance. In this paper, we aim to improve the effectiveness of pointwise methods while preserving their efficiency through two key innovations: (1) We propose a novel Global-Consistent Comparative Pointwise Ranking (GCCP) strategy that incorporates global reference comparisons between each candidate and an anchor document to generate contrastive relevance scores. We strategically design the anchor document as a query-focused summary of pseudo-relevant candidates, which serves as an effective reference point by capturing the global context for document comparison. (2) These contrastive relevance scores can be efficiently Post-Aggregated with existing pointwise methods, seamlessly integrating essential Global Context information in a training-free manner (PAGC). Extensive experiments on the TREC DL and BEIR benchmark demonstrate that our approach significantly outperforms previous pointwise methods while maintaining comparable efficiency. Our method also achieves competitive performance against comparative methods that require substantially more computational resources. More analyses further validate the efficacy of our anchor construction strategy.
Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval
Authors: Zijun Long, Kangheng Liang, Gerardo Aragon-Camarasa, Richard McCreadie, Paul Henderson
Venue: SIGIR
Year: 2025

Interactive Text-to-image retrieval (I-TIR) is an important enabler for a wide range of state-of-the-art services in domains such as e-commerce and education. However, current methods rely on finetuned Multimodal Large Language Models (MLLMs), which are costly to train and update, and exhibit poor generalizability. This latter issue is of particular concern, as: 1) finetuning narrows the pretrained distribution of MLLMs, thereby reducing generalizability; and 2) I-TIR introduces increasing query diversity and complexity. As a result, I-TIR solutions are highly likely to encounter queries and images not well represented in any training dataset. To address this, we propose leveraging Diffusion Models (DMs) for text-to-image mapping, to avoid finetuning MLLMs while preserving robust performance on complex queries. Specifically, we introduce Diffusion Augmented Retrieval (DAR), a framework that generates multiple intermediate representations via LLM-based dialogue refinements and DMs, producing a richer depiction of the user's information needs. This augmented representation facilitates more accurate identification of semantically and visually related images. Extensive experiments on four benchmarks show that for simple queries, DAR achieves results on par with finetuned I-TIR models, yet without incurring their tuning overhead. Moreover, as queries become more complex through additional conversational turns, DAR surpasses finetuned I-TIR models by up to 7.61% in Hits@10 after ten turns, illustrating its improved generalization for more intricate queries.
Is Having Rationales Enough? Rethinking Knowledge Enhancement for Multimodal Hateful Meme Detection
Authors: Junyu Lu, Bo Xu, Xiaokun Zhang, Haohao Zhu, Kaichun Wang, Liang Yang, Hongfei Lin
Venue: SIGIR
Year: 2025

Hateful memes are prevalent on the Internet, raising the urgent need for effective detection.Given their implicit nature, incorporating rationales with background knowledge is crucial for enhancing model understanding.However, existing methods often suffer from limited quality of external rationales and misalignment with original meme information.These challenges hinder model comprehension, leading to reduced accuracy and explainability.To address these challenges, we propose a Multimodal Multi-agent Knowledge Enhanced (M2KE) framework for hateful meme detection.M2KE introduces a multi-agent rationale discovery mechanism to extract high-quality rationales relevant to meme content and an adaptive knowledge interaction mechanism to ensure alignment between original meme information and external rationales.Specifically, multi-agent rationale discovery mechanism improves the reliability of rationales by collaboratively verifying and refining them with multiple agents, supported by large language models (LLMs) due to their extensive knowledge.And adaptive knowledge interaction mechanism uses information entropy to dynamically balance the model's attention between original meme information and external rationales, preventing over-reliance on rationales and enabling a more comprehensive understanding.Experimental results on three datasets demonstrate that M2KE significantly outperforms existing models.Further analysis underscores the importance of effectively integrating accurate rationales to enhance model performance.
Dual Debiasing in LLM-based Recommendation
Authors: Sijin Lu, Zhibo Man, Fangyuan Luo, Jun Wu
Venue: SIGIR
Year: 2025

Large language models (LLMs) have been widely applied in recommender systems, achieving remarkable success. However, LLM-based recommendation (LR) suffers from more severe popularity bias than conventional recommendation (CR), stemming from both training and inference stages. In this paper, we propose a novel debiasing method for LR, which performs debiasing in such two stages, so termed as Dual Debiasing in LR (D²LR). Concretely, in the training stage, we conduct token-wise inverse propensity score weighting to force the LLM to pay more attention on unpopular tokens. In the inference stage, we train a more biased CR model by increasing the weights of popular items, which adjusts the generation probability of corresponding tokens according to its scores for items, hoping to suppress the excessive generation of popular tokens. Experiments conducted on three real-world datasets validate the effectiveness of our D²LR in mitigating popularity bias in LR.
PILs of Knowledge: A Synthetic Benchmark for Evaluating Question Answering Systems in Healthcare
Authors: Riccardo Lunardi, Michael Soprano, Paolo Coppola, Vincenzo Della Mea, Stefano Mizzaro, Kevin Roitero
Venue: SIGIR
Year: 2025

Patient Information Leaflets (PILs) provide essential information about medication usage, side effects, precautions, and interactions, making them a valuable resource for Question Answering (QA) systems in healthcare. However, no dedicated benchmark currently exists to evaluate QA systems specifically on PILs, limiting progress in this domain. To address this gap, we introduce a fact-supported synthetic benchmark composed of multiple-choice questions and answers generated from real PILs. We construct the benchmark using a fully automated pipeline that leverages multiple Large Language Models (LLMs) to generate diverse, realistic, and contextually relevant question-answer pairs. The benchmark is publicly released as a standardized evaluation framework for assessing the ability of LLMs to process and reason over PIL content. To validate its effectiveness, we conduct an initial evaluation with state-of-the-art LLMs, showing that the benchmark presents a realistic and challenging task, making it a valuable resource for advancing QA research in the healthcare domain.
Language Model Alignment for Conversational Shopping at Amazon
Authors: Chen Luo, Dimitri Papadimitriou, Hariharan Muralidharan, Dhineshkumar Ramasubbu, Aakash Kolekar, Wenju Xu, Cong Xu, Anirudh Srinivasan, Mukesh Jain, Qi He
Venue: SIGIR
Year: 2025

The rapid growth of online shopping stores, such as Amazon, has led to services reaching billions of people worldwide. With global retail sales exceeding $6 trillion in 2024, customer expectations for personalized and seamless shopping experiences have heightened. Traditional online shopping experiences, such as search and navigation systems, often fall short in addressing complex shopping journeys. Conversational shopping (such as Amazon Rufus) offers a transformative approach by enabling dynamic, multi-turn dialogues that closely resemble human interactions. This allows customers to explore product options, seek clarifications, and receive personalized recommendations, thereby enhancing product discovery and informed decision-making. In this paper, we share our year-long journey of using language models for conversational shopping at Amazon and introduce how we use LLM fine-tuning techniques to enhance LLMs for a conversational shopping experience like Amazon Rufus. We also introduce innovative strategies for training data collection and demonstrate real-world applications, including product recommendations, clarification mechanisms, and internationalization for global customers.
CDC: Causal Domain Clustering for Multi-Domain Recommendation
Authors: Huishi Luo, Yiqing Wu, Yiwen Chen, Fuzhen Zhuang, Deqing Wang
Venue: SIGIR
Year: 2025

Multi-domain recommendation leverages domain-general knowledge to improve recommendations across several domains. However, as platforms expand to dozens or hundreds of scenarios, training all domains in a unified model leads to performance degradation due to significant inter-domain differences. Existing domain grouping methods, based on business logic or data similarities, often fail to capture the true transfer relationships required for optimal grouping. To effectively cluster domains, we propose Causal Domain Clustering (CDC). CDC models domain transfer patterns within a large number of domains using two distinct effects: the Isolated Domain Affinity Matrix for modeling non-interactive domain transfers, and the Hybrid Domain Affinity Matrix for considering dynamic domain synergy or interference under joint training. To integrate these two transfer effects, we introduce causal discovery to calculate a cohesion-based coefficient that adaptively balances their contributions. A Co-Optimized Dynamic Clustering algorithm iteratively optimizes target domain clustering and source domain selection for training. CDC significantly enhances performance across over 50 domains on public datasets and in industrial settings, achieving a 4.9% increase in online eCPM. Code is available online: https://github.com/Chrissie-Law/Causal-Domain-Clustering-for-Multi-Domain-Recommendation.
PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation
Authors: Chenglong Ma, Ziqi Xu, Yongli Ren, Danula Hettiachchi, Jeffrey Chan
Venue: SIGIR
Year: 2025

Traditional offline evaluation methods for recommender systems struggle to capture the complexity of modern platforms due to sparse behavioural signals, noisy data, and limited modelling of user personality traits. While simulation frameworks can generate synthetic data to address these gaps, existing methods fail to replicate behavioural diversity, limiting their effectiveness. To overcome these challenges, we propose the Personality-driven User Behaviour Simulator (PUB), an LLM-based simulation framework that integrates the Big Five personality traits to model personalised user behaviour. PUB dynamically infers user personality from behavioural logs (e.g., ratings, reviews) and item metadata, then generates synthetic interactions that preserve statistical fidelity to real-world data. Experiments on the Amazon review datasets show that logs generated by PUB closely align with real user behaviour and reveal meaningful associations between personality traits and recommendation outcomes. These results highlight the potential of the personality-driven simulator to advance recommender system evaluation, offering scalable, controllable, high-fidelity alternatives to resource-intensive real-world experiments.
Graph Isomorphism Network-Based Cohort Modeling In Click-Through Rate Prediction
Authors: Xuan Ma, Hao Peng, Jia Duan, Zhanhao Ye, Langlang Ye, Zehua Zhang, Jie He, Changping Peng, Zhangang Lin
Venue: SIGIR
Year: 2025

Accurate Click-Through Rate (CTR) prediction is vital for search engines and recommendation systems, yet it is often hindered by the ''cold start problem'', which arises from insufficient historical data for new users. Recent approaches have sought to tackle this by training encoder-decoder networks on data from warm users to generate virtual behavior embeddings for cold users. However, these methods have shortcomings in terms of simplistic encoding techniques for warm user behaviors and direct utilization of virtual behavior embeddings, leading to limitations in user interest expression and generalization. To address these challenges, we propose a novel method that leverages Graph Isomorphism Networks (GIN) for cohort modeling within CTR prediction. GIN effectively captures high-order user-item interactions, providing a more nuanced understanding of users' diverse interests. Additionally, the cohort modeling strategy minimizes deviations in constructed embeddings, enhancing the model's generalization abilities. We validate our approach through experiments on public and industrial datasets, demonstrating significant improvements for both warm and cold users compared to existing methodologies. Furthermore, we implemented the GIN Cohort Modeling (GINCM) in a large-scale online advertising system, optimizing for both pre-computation and real-time processing to reduce latency. The implementation yields notable enhancements of 2.13% in CTR and Revenue Per Mille(RPM), showcasing the practical effectiveness and real-world applicability of our model.
Generating Difficulty-aware Negative Samples via Conditional Diffusion for Multi-modal Recommendation
Authors: Wenze Ma, Chenyu Sun, Yanmin Zhu, Zhaobo Wang, Xuhao Zhao, Mengyuan Jing, Jiadi Yu, Feilong Tang
Venue: SIGIR
Year: 2025

Designing effective negative sampling strategies is crucial for training Multi-Modal Recommendation (MMRec) models, as it helps address the issues of sparse user-item interactions and facilitates the learning of high-dimensional modality features. However, most existing methods randomly sample non-interacted items as negative ones, which frequently result in easy negatives. They limit the model's ability to accurately capture user preferences. In this paper, we propose to Generate Difficulty-aware Negative Samples via conditional diffusion for MMRec (denoted as GDNSM). Leveraging the rich semantic and contextual information from multi-modal features, our method generates hard negative samples with varying difficulty levels, tailored to user preferences. They force the model to learn finer-grained distinctions between positive and negative samples, enhancing its ability to inferring user preferences. To avoid unstable training, we design a dynamic difficulty scheduling mechanism that schedules the negative samples from easy to hard for model training, ensuring both stability and effectiveness. Extensive experiments on three real-world datasets demonstrate that the effectiveness of our models.
Reconciling Efficiency and Effectiveness of Exercise Retreival: An Uncertainty Reduction Hashing Approach for Computerized Adaptive Testing
Authors: Haiping Ma, Weiyuan Zhou, Xiaoshan Yu, Changqian Wang, Shangshang Yang, Limiao Zhang, Xingyi Zhang
Venue: SIGIR
Year: 2025

With the rapid development of intelligent education, Computerized Adaptive Testing(CAT) has garnered significant attention for its ability to tailor exercises to individual examinees. The adaptability of CAT is primarily achieved through the alternating optimization of two core components: the cognitive diagnosis model and the exercise selection module. However, existing CAT approaches, despite their remarkable achievements, often come at the expense of high time costs. Statistical-based approaches incur increased time overhead due to complex computations, while data-driven approaches further exacerbate time inefficiency because of the iterative processes in reinforcement learning, making it challenging to balance evaluation effectiveness and time efficiency. To this end, in this paper, we propose HashCAT, an efficient CAT approach based on learning to hash, aiming to balance efficiency and evaluation effectiveness. Our approach comprises two stages: the hash representation generation and the exercise selection. In the first stage, we design an information alignment module and a novel cognitive diagnosis function to model the interaction between examinees and exercises, generating hash representations with clear physical significance. In the second stage, we propose an uncertainty reduction-based algorithm that utilize information entropy to quantify the uncertainty in student ability estimation and selects exercises that most effectively reduce this uncertainty. Experimental results on four real-world datasets demonstrate that the proposed method significantly improves question selection efficiency while maintaining competitive evaluation performance. The code exists anonymously in https://github.com/sherklock/Intelligent-Education/tree/main/HashCAT-main.
Artifact Sharing for Information Retrieval Research
Authors: Sean MacAvaney
Venue: SIGIR
Year: 2025

Sharing artifacts-such as trained models, pre-built indexes, and the code to use them-aids in reproducibility efforts by allowing researchers to validate intermediate steps and improves the sustainability of research by allowing multiple groups to build off one another's prior computational work. Although there are de facto consensuses on how to share research code (through a git repository linked to from publications) and trained models (via HuggingFace Hub), there is no consensus for other types of artifacts, such as built indexes. Given the practical utility of using shared indexes, researchers have resorted to self-hosting these resources or performing ad hoc file transfers upon request, ultimately limiting the artifacts' discoverability and reuse. This demonstration introduces a flexible and interoperable way to share artifacts for Information Retrieval research, improving both their accessibility and usability.
Constructing and Evaluating Declarative RAG Pipelines in PyTerrier
Authors: Craig Macdonald, Jinyuan Fang, Andrew Parry, Zaiqiao Meng
Venue: SIGIR
Year: 2025

Search engines often follow a pipeline architecture, where complex but effective reranking components are used to refine the results of an initial retrieval. Retrieval augmented generation (RAG) is an exciting application of the pipeline architecture, where the final component generates a coherent answer for the users from the retrieved documents. In this demo paper, we describe how such RAG pipelines can be formulated in the declarative PyTerrier architecture, and the advantages of doing so. Our PyTerrier-RAG extension for PyTerrier provides easy access to standard RAG datasets and evaluation measures, state-of-the-art LLM readers, and using PyTerrier's unique operator notation, easy-to-build pipelines. We demonstrate the succinctness of indexing and RAG pipelines on standard datasets (including Natural Questions) and how to build on the larger PyTerrier ecosystem with state-of-the-art sparse, learned-sparse, and dense retrievers, and other neural rankers.