OpenRaiser/Intern-Atlas huggingface dataset:
A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models
Authors: Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, Guido Zuccon
Venue: SIGIR
Year: 2024

We propose a novel zero-shot document ranking approach based on Large Language Models (LLMs): the Setwise prompting approach. Our approach complements existing prompting approaches for LLM-based zero-shot ranking: Pointwise, Pairwise, and Listwise. Through the first-of-its-kind comparative evaluation within a consistent experimental framework and considering factors like model size, token consumption, latency, among others, we show that existing approaches are inherently characterised by trade-offs between effectiveness and efficiency. We find that while Pointwise approaches score high on efficiency, they suffer from poor effectiveness. Conversely, Pairwise approaches demonstrate superior effectiveness but incur high computational overhead. Our Setwise approach, instead, reduces the number of LLM inferences and the amount of prompt token consumption during the ranking procedure, compared to previous methods. This significantly improves the efficiency of LLM-based zero-shot ranking, while also retaining high zero-shot ranking effectiveness. We make our code and results publicly available at https://github.com/ielab/llm-rankers.
Reflections on the Coding Ability of LLMs for Analyzing Market Research Surveys
Authors: Shi Zong, Santosh Kolagati, Amit Chaudhary, Josh Seltzer, Jimmy Lin
Venue: SIGIR
Year: 2024

The remarkable success of large language models (LLMs) has drawn people's great interest in their deployment in specific domains and downstream applications. In this paper, we present the first systematic study of applying large language models (in our case, GPT-3.5 and GPT-4) for the automatic coding (multi-class classification) problem in market research. Our experimental results show that large language models could achieve a macro F1 score of over 0.5 for all our collected real-world market research datasets in a zero-shot setting. We also provide in-depth analyses of the errors made by the large language models. We hope this study sheds light on the lessons we learn and the open challenges large language models have when adapting to a specific market research domain.
Adaptive Structure Learning with Partial Parameter Sharing for Post-Click Conversion Rate Prediction
Authors: Chunyuan Zheng, Hang Pan, Yang Zhang, Haoxuan Li
Venue: SIGIR
Year: 2025

The post-click conversion rate (CVR) prediction task aims to predict the probability of a conversion after a click, which is essential in many fields. There are two widely-recognized challenges for CVR prediction: selection bias and data sparsity. Many previous methods focus on addressing selection bias by unbiasedly estimating the ideal loss based on the doubly robust estimator, which incorporates the error imputation model and propensity model to help CVR prediction model learning. However, they struggle with unreasonable knowledge transfer between the prediction model and imputation model and inflexible network structure design under sparse data. To this end, we introduce a novel principled adaptive structure learning approach, named Adap-SL, to adaptively learn the optimal network structure, adjust the number of activated (non-zero) parameters, and determine which knowledge needs to be transferred between the prediction model and the imputation model. Specifically, we start with an over-parameterized base network, where we adaptively extract partially overlapped subnetworks for the imputation model and the prediction model. Extensive experiments are conducted on three real-world recommendation datasets, demonstrating that our method consistently improves performance while requiring fewer parameters. The code is available at https://github.com/ChunyuanZheng/sigir25-sparse-sharing.
Linear Item-Item Models with Neural Knowledge for Session-based Recommendation
Authors: Minjin Choi, Sunkyung Lee, Seongmin Park, Jongwuk Lee
Venue: SIGIR
Year: 2025

Session-based recommendation (SBR) aims to predict users' subsequent actions by modeling short-term interactions within sessions. Existing neural models primarily focus on capturing complex dependencies for sequential item transitions. As an alternative solution, linear item-item models mainly identify strong co-occurrence patterns across items and support faster inference speed. Although each paradigm has been actively studied in SBR, their fundamental differences in capturing item relationships and how to bridge these distinct modeling paradigms effectively remain unexplored. In this paper, we propose a novel SBR model, namely Linear Item-Item model with Neural Knowledge (LINK), which integrates both types of knowledge into a unified linear framework. Specifically, we design two specialized components of LINK: (i) Linear knowledge-enhanced Item-item Similarity model (LIS), which refines the item similarity correlation via self-distillation, and (ii) Neural knowledge-enhanced Item-item Transition model (NIT), which seamlessly incorporates complicated neural knowledge distilled from the off-the-shelf neural model. Extensive experiments demonstrate that LINK outperforms state-of-the-art linear SBR models across six real-world datasets, achieving improvements of up to 14.78% and 11.04% in Recall@20 and MRR@20 while showing up to 813x fewer inference FLOPs. Our code is available at https://github.com/jin530/LINK.
GEAR: Generalized Alternating Regressor for Multi-Behavior Sequential Recommendation
Authors: Junzhe Jiang, Kai Zhang, Junfeng Kang, Yucong Luo, Min Gao
Venue: SIGIR
Year: 2025

Modern recommender systems face a critical challenge in modeling the intricate interplay between multi-behavior interactions of users (e.g., clicks, adds-to-cart and purchases) and temporal dynamics that drive evolving preferences. While existing multi-behavior sequential recommendation methods attempt to capture these signals, they often suffer from fragmented modeling, such as decoupling behaviors and items into separate sequences, neglecting time-aware transitions, or relying on computationally intensive architectures that hinder real-world scalability. To address these limitations, we propose GEneralized Alternating Regressor (GEAR), a novel framework that unifies behaviors, items, and temporal contexts into a single autoregressive sequence through an alternating architecture. At its core, GEAR represents user interactions as triplets and processes them through a modular transformer architecture. In this architecture, each triplet is alternately modeled at lower layers to disentangle fine-grained patterns, while upper layers jointly learn cross-signal dependencies. This design mimics the interlocking mechanism of gears, enabling the seamless transitions between multi-behavior dynamics and item transitions. Additionally, we incorporate a time-bias term to quantify the decay of behavioral influence across both short- and long-term horizons. Extensive experiments on real-world datasets validate the effectiveness, generalizability, and computational efficiency of the proposed framework.
Navigating Large Language Models for Recommendation: From Architecture to Learning Paradigms and Deployment
Authors: Xinyu Lin, Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng
Venue: SIGIR
Year: 2025

Large Language Models (LLMs) are reshaping the landscape of recommender systems, giving rise to the emerging field of LLM4Rec that attracts both academia and industry. Unlike earlier approaches that simply borrowed model architectures or learning paradigms from language models, recent advances have led to a dedicated and evolving technical stack for LLM4Rec, spanning architecture design, pre-training and post-training strategies, inference techniques, and real-world deployment. This tutorial offers a systematic and in-depth overview of LLM4Rec through the lens of this technical stack. We will examine how LLMs are being adapted to recommendation tasks across different stages, empowering them with capabilities such reasoning, planning, and in-context learning. Moreover, we will highlight practical challenges including complex user modeling, trustworthiness, and evaluation. Distilling insights from recent research and identifying open problems, this tutorial aims to equip participants with a comprehensive understanding of LLM4Rec and inspire continued innovation in this rapidly evolving field.
Revolutionizing Text-to-Image Retrieval as Autoregressive Token-to-Voken Generation
Authors: Yongqi Li, Hongru Cai, Wenjie Wang, Leigang Qu, Yinwei Wei, Wenjie Li, Liqiang Nie, Tat-Seng Chua
Venue: SIGIR
Year: 2025

Text-to-image retrieval is a fundamental task in multimedia retrieval. Traditional studies have typically approached this task as a discriminative problem, matching the text and image via the cross-attention mechanism (one-tower framework) or in a common embedding space (two-tower framework). The one-tower framework excels in effectiveness but falls short in efficiency, whereas the two-tower framework is efficient but struggles to maintain competitive effectiveness. In this study, we aim to enhance both effectiveness and efficiency by transforming the text-to-image retrieval task into a token-to-voken generation problem, where fine-grained interactions are incorporated to improve effectiveness while maintaining high efficiency. Despite its potential advantages, this paradigm shift presents significant challenges: 1) misalignment with high-level semantics and 2) learning gap towards the retrieval target. To address the challenges, we propose AVG, which discretizes images into vokens while aligning with both the visual information and high-level semantics. Additionally, to bridge the learning gap between generative training and the retrieval target, AVG incorporates discriminative training to modify the learning direction during token-to-voken training. Experiments demonstrate that the benefits of paradigm innovation are realized: compared with the classical two-tower method, CLIP, AVG achieves the 7.53% relative effectiveness improvement and also 4× efficiency improvement.
The Second Tutorial on Retrieval-Enhanced Machine Learning: Synthesis and Opportunities
Authors: Fernando Diaz, Andrew Drozdov, To Eun Kim, Alireza Salemi, Hamed Zamani
Venue: SIGIR
Year: 2025

Retrieval-Enhanced Machine Learning (REML) refers to the use of information retrieval (IR) methods to support reasoning and inference in machine learning tasks. Although relatively recent, these approaches can substantially improve model performance. This includes improved generalization, knowledge grounding, scalability, freshness, attribution, interpretability, and on-device learning. To date, despite being influenced by work in the information retrieval community, REML research has predominantly been presented in natural language processing (NLP) conferences. Our tutorial addresses this disconnect by introducing core REML concepts and synthesizing the literature from various domains in machine learning (ML), including, but not limited to, NLP. What is unique to our approach is the use of consistent notations to provide researchers with a unified and expandable framework. The tutorial will be presented in lecture format based on an existing manuscript, with supporting materials and a comprehensive reading list available at a website. Building on the momentum of our successful workshop at SIGIR 2023 and our tutorial at SIGIR-AP 2024, this year's tutorial features updated content with an emphasis on retrieval technologies used across the broader ML community. We also highlight their role in emerging, future-facing applications such as language agents and evolving scenarios where the extensive body of knowledge from IR can provide critical insights and capabilities.
Exploring the Escalation of Source Bias in User, Data, and Recommender System Feedback Loop
Authors: Yuqi Zhou, Sunhao Dai, Liang Pang, Gang Wang, Zhenhua Dong, Jun Xu, Ji-Rong Wen
Venue: SIGIR
Year: 2025

Recommender systems are essential for information access, allowing users to present their content for recommendation. With the rise of large language models (LLMs), AI-generated content (AIGC), primarily in the form of text, has become a central part of the content ecosystem. As AIGC becomes increasingly prevalent, it is important to understand how it affects the performance and dynamics of recommender systems. To this end, we construct an environment that incorporates AIGC to explore its short-term impact. The results from popular sequential recommendation models reveal that AIGC are ranked higher in the recommender system, reflecting the phenomenon of source. To further explore the long-term impact of AIGC, we introduce a feedback loop with realistic simulators. The results show that the model's preference for AIGC increases as the user clicks on AIGC rises and the model trains on simulated click data. This leads to two issues: In the short term, bias toward AIGC encourages LLM-based content creation, increasing AIGC content, and causing unfair traffic distribution. From a long-term perspective, our experiments also show that when AIGC dominates the content ecosystem after a feedback loop, it can lead to a decline in recommendation performance. To address these issues, we propose a debiasing method based on L1-loss optimization to maintain long-term content ecosystem balance. In a real-world environment with AIGC generated by mainstream LLMs, our method ensures a balance between AIGC and human-generated content in the ecosystem. The code and dataset are available at https://github.com/Yuqi-Zhou/Rec_SourceBias.
ReNeuIR at SIGIR 2025: The Fourth Workshop on Reaching Efficiency in Neural Information Retrieval
Authors: Sebastian Bruch, Maik Fröbe, Tim Hagen, Franco Maria Nardini, Martin Potthast
Venue: SIGIR
Year: 2025

Measuring effectiveness and efficiency in information retrieval has a strong empirical background. While modern retrieval systems substantially improve effectiveness, the community has not yet agreed on how to measure efficiency, making it difficult to contrast effectiveness and efficiency fairly. Efficiency-oriented system comparisons are difficult due to factors such as hardware configurations, software versioning, and experimental settings. Efficiency affects users, researchers, and the environment and can be measured in many dimensions beyond time and space, such as resource consumption, water usage, and sample efficiency. Analyzing the efficiency of algorithms and their trade-off with effectiveness requires revisiting and establishing new standards and principles, from defining relevant concepts to designing new measures and guidelines to assess the findings' significance. ReNeuIR's fourth iteration aims to bring the community together to debate these questions and collaboratively test and improve benchmarking frameworks for efficiency based on discussions and collaborations of its previous iterations, including a shared task focused on efficiency and reproducibility.
You Are What You Bought: Generating Customer Personas for E-commerce Applications
Authors: Yimin Shi, Yang Fei, Shiqi Zhang, Haixun Wang, Xiaokui Xiao
Venue: SIGIR
Year: 2025

In e-commerce, user representations are essential for various applications. Existing methods often use deep learning techniques to convert customer behaviors into implicit embeddings. However, these embeddings are difficult to understand and integrate with external knowledge, limiting the effectiveness of applications such as customer segmentation, search navigation, and product recommendations. To address this, our paper introduces the concept of the customer persona. Condensed from a customer's numerous purchasing histories, a customer persona provides a multi-faceted and human-readable characterization of specific purchase behaviors and preferences, such as Busy Parents or Bargain Hunters. This work then focuses on representing each customer by multiple personas from a predefined set, achieving readable and informative explicit user representations. To this end, we propose an effective and efficient solution GPLR. To ensure effectiveness, GPLR leverages pre-trained LLMs and few-shot learning to infer personas for customers. To reduce overhead, GPLR applies LLM-based labeling to only a fraction of users and utilizes a random walk technique to predict personas for the remaining customers. To further enhance efficiency, we propose an approximate solution called RevAff for this random walk-based computation. RevAff provides an absolute error ε guarantee while improving the time complexity of the exact solution by a factor of at least O( ε ⋅|E|N over |E| + N log N ), where N represents the number of customers and products, and E represents the interactions between them. We evaluate the performance of our persona-based representation in terms of accuracy and robustness for recommendation and customer segmentation tasks using three real-world e-commerce datasets. Most notably, we find that integrating customer persona representations improves the state-of-the-art graph convolution-based recommendation model by up to 12% in terms of NDCG@K and F1-Score@K.
Modeling Social Behavior in Collaborative Filtering
Authors: Yihong Zhang, Takahiro Hara
Venue: SIGIR
Year: 2025

Nowadays, many online services use recommendation systems to provide personalized item recommendations to users. Collaborative filtering is the major paradigm in recommendation systems. Based on user-item interaction data, collaborative filtering recommends items to a user based on other similar users. The problem of interest disentanglement in recommendation now has attracted the attention of many researchers. Several works have proposed methods to disentangle conformity from user private interest, by assuming that conformity is correlated to item popularity. However, such modeling is simplistic and overlooks many possibilities between user public and private interest, and the item popularity. For example, a user can privately like a popular movie or buy a niche music album due to the stimulation of the social environment. In this paper, we propose a more comprehensive social behavior model that describes fine-grained relationships between user interest and item popularity. Our model does not use explicit user relationship data. Instead, we extract social behavior patterns directly from user-item interaction data. We also make our model into a recommendation framework called Disentangled Social Consumer Preference (DSCP), which can be integrated into existing recommendation models such as BPRMF. Our extensive experiments with four datasets from different services show that our model can outperform state-of-the-art baseline models. We achieve better recommendation accuracy in both the usual random test and the intervened test that shows debiasing effect.
SIGIR 2025 Workshop on eCommerce (ECOM25): From Research to Product: Challenges, Lessons, and Opportunities in eCommerce Search and Recommendations
Authors: Yubin Kim, Tracy Holloway King, Aditya Chichani, Pallavi Gudipati, Andrew Trotman
Venue: SIGIR
Year: 2025

The eCommerce search and recommendations space is a unique and dynamic domain within information retrieval (IR), characterized by its multimodality and industry-driven challenges. While the basic task of fulfilling a user's information need aligns with web search, the methodologies employed are distinct. On eCommerce platforms (e.g. Alibaba, Amazon, eBay, Etsy, Flipkart, Walmart), the data available for retrieval and ranking differs significantly, as do the success signals (e.g. adding items to a cart, purchasing). Our focus for 2025 is on fostering deeper engagement through interactive discussions, exploring crucial topics such as navigating irreproducibility in research-to-product pipelines, and addressing emerging topics such as evaluation metrics for LLMs, multimodality, and the interplay between organic and sponsored search. With our discussion-heavy format and structured facilitation, we aim to spark conversation among all participants.
Bridging Interests and Truth: Towards Mitigating Fake News with Personalized and Truthful Recommendations
Authors: Zihan Ma, Minnan Luo, Yiran Hao, Zhi Zeng, Xiangzheng Kong, Jiahao Wang
Venue: SIGIR
Year: 2025

While the proliferation of fake news poses a significant threat to information integrity, existing efforts to counter it, especially within personalized news recommendation systems, have proven inadequate. Traditional methods, which often rely on classifiers to filter out fake content, are limited by their accuracy and their inability to fully capture the diverse interests of users. To address these challenges, we proposed PRISM --- Protection-enhanced Recommendation with Interest-aware Sequential Modeling --- a novel framework based on diffusion models. PRISM harnesses the generative and control capabilities of diffusion models to progressively learn the implicit distribution of user interests from their reading history, thereby generating personalized recommendations that align with both their linguistic preferences and interest domains. Furthermore, PRISM incorporates pre-trained authenticity representations as constraints during content generation, ensuring the credibility of the recommended news and effectively curbing the spread of fake news. Comprehensive evaluations from multiple dimensions demonstrate the superiority of our model.
CLIP-AdaM: Adapting Multi-view CLIP for Open-set 3D Object Retrieval
Authors: Xinwei He, Liang Ma, Yuxuan Cheng, Zhichuan Wang, Yulong Wang, Yang Zhou, Xiang Bai
Venue: SIGIR
Year: 2025

Open-set 3D object retrieval (3DOR) aims to learn discriminative and generalizable embeddings for unseen categories of 3D objects. However, attaining this objective typically requires the costly acquisition of large-scale 3D object datasets and associated resources for model training. Building upon the strong open-world representation capabilities of CLIP, we introduce CLIP-AdaM, which, to our knowledge, represents the first attempt to adapt a CLIP model for open-set 3DOR with minimal effort. We first find that a pretrained CLIP already delivers a surprisingly acceptable performance on multi-view images. To further unleash its potential, we design a customized adapter for learning to aggregate and adapt its pretrained features towards better 3D embeddings. For aggregation, it learns two sets of view scores to weigh the contributions of view images for fusion. One is learned by a tiny view-score network at the instance level, and the other is learned implicitly at the dataset level, aiding generalization to unseen categories. The adaptation component comprises only a basic linear layer yet yields superior results. During training, the adapter with such a small amount of parameters can be efficiently fine-tuned with limited 3D closed-set data, effectively mitigating the overfitting issue while harnessing the prior knowledge from pretrained models. Without bells and whistles, CLIP-AdaM attains state-of-the-art performance on four open-set 3DOR benchmarks. Additionally, it demonstrates strong extensibility to broader scenarios, including zero-shot, few-shot, and seen/unseen 3D representation learning.