OpenRaiser/Intern-Atlas huggingface dataset:
Generative Auto-Bidding with Value-Guided Explorations
Authors: Jingtong Gao, Yewen Li, Shuai Mao, Peng Jiang, Nan Jiang, Yejing Wang, Qingpeng Cai, Fei Pan, Peng Jiang, Kun Gai, Bo An, Xiangyu Zhao
Venue: SIGIR
Year: 2025

Auto-bidding, with its strong capability to optimize bidding decisions within dynamic and competitive online environments, has become a pivotal strategy for advertising platforms. Existing approaches typically employ rule-based strategies or Reinforcement Learning (RL) techniques. However, rule-based strategies lack the flexibility to adapt to time-varying market conditions, and RL-based methods struggle to capture essential historical dependencies and observations within Markov Decision Process (MDP) frameworks. Furthermore, these approaches often face challenges in ensuring strategy adaptability across diverse advertising objectives. Additionally, as offline training methods are increasingly adopted to facilitate the deployment and maintenance of stable online strategies, the issues of documented behavioral patterns and behavioral collapse resulting from training on fixed offline datasets become increasingly significant. To address these limitations, this paper introduces a novel offline Generative Auto-bidding framework with Value-Guided Explorations (GAVE). GAVE accommodates various advertising objectives through a score-based Return-To-Go (RTG) module. Moreover, GAVE integrates an action exploration mechanism with an RTG-based evaluation method to explore novel actions while ensuring stability-preserving updates. A learnable value function is also designed to guide the direction of action exploration and mitigate Out-of-Distribution (OOD) problems. Experimental results on two offline datasets and real-world deployments demonstrate that GAVE outperforms state-of-the-art baselines in both offline evaluations and online A/B tests. By applying the core methods of this framework, we proudly secured first place in the NeurIPS 2024 competition, 'AIGB Track: Learning Auto-Bidding Agents with Generative Models'.
Social Context-Aware Community-Level Propagation Prediction
Authors: Jinfei Gao, Xiao Wang, Tian Gan, Jianhua Yin, Chuanchen Luo, Liqiang Nie
Venue: SIGIR
Year: 2025

With the increasing prevalence of online communities, social networks have become pivotal platforms for information propagation. However, this rise is accompanied by issues such as the spread of misinformation and online rumors. Community Level Information Pathway Prediction (CLIPP) is proposed to effectively stop the propagation of harmful information within specific communities. While progress has been made in understanding user-level propagation, there is a significant gap in addressing the CLIPP problem at the community level, particularly with regard to social context interpretation and the cold start problem in niche communities. To bridge this gap, we propose a novel model, named Community-Level Propagation Prediction with LLM enhanced Social Context Interpretation and Community Coldstart (ComPaSC3), which integrates three primary modules. The video enhancement module leverages LLMs to enrich the interpretation of multimedia content by embedding world knowledge. The community portrait building module utilizes LLMs to generate detailed community portraits for community interpretation. To tackle the community cold start problem, the dynamic commLink module links non-popular communities to the popular ones based on their portrait similarity, and dynamically updates their relationship weights. Our experimental results demonstrate that ComPaSC3 significantly improves predictive accuracy in both popular and non-popular scenarios. Particularly in non-popular communities, our approach outperforms existing state-of-the-art methods, achieving improvements of 10.00% - 15.20% in Rec@5 and 7.31% - 12.32% in NDCG@10.
Towards Improving Image Quality in Second-Hand Marketplaces with LLMs
Authors: Sandra Garcia-Esparza, Victor Codina, Salma Lahbiss, Imène Sediri, Nafi Cissé
Venue: SIGIR
Year: 2025

The quality of image products in second-hand marketplaces is a critical factor as it can significantly impact a buyer's decision-making process as well as overall user trust. Inspired by the successful application of LLM capabilities as text-based task evaluators, we propose an approach that leverages multi-modal large language models (MLLMs) as evaluators of image quality. In this work, we conduct a systematic comparison of several state-of-the-art MLLMs, evaluating the alignment between the scores generated by these models and the human scores collected from a survey of 929 users in a second-hand marketplace. Our findings demonstrate that some of the evaluated MLLMs can achieve a high level of agreement with human judgments. To further understand the differences between LLM-based and human scores, we also present an analysis of the alignment between explanations generated by LLMs and those provided by humans. Overall, we believe our findings underscore the potential of LLMs for automatic image quality assessment.
Information Retrieval in the Age of Generative AI: The RGB Model
Authors: Michele Garetto, Alessandro Cornacchia, Franco Galante, Emilio Leonardi, Alessandro Nordio, Alberto Tarable
Venue: SIGIR
Year: 2025

The advent of Large Language Models (LLMs) and generative AI is fundamentally transforming information retrieval and processing on the Internet, bringing both great potential and significant concerns regarding content authenticity and reliability. This paper presents a novel quantitative approach to shed light on the complex information dynamics arising from the growing use of generative AI tools. Despite their significant impact on the digital ecosystem, these dynamics remain largely uncharted and poorly understood. We propose a stochastic model to characterize the generation, indexing, and dissemination of information in response to new topics. This scenario particularly challenges current LLMs, which often rely on real-time Retrieval-Augmented Generation (RAG) techniques to overcome their static knowledge limitations. Our findings suggest that the rapid pace of generative AI adoption, combined with increasing user reliance, can outpace human verification, escalating the risk of inaccurate information proliferation across digital resources. An in-depth analysis of Stack Exchange data confirms that high-quality answers inevitably require substantial time and human effort to emerge. This underscores the considerable risks associated with generating persuasive text in response to new questions and highlights the critical need for responsible development and deployment of future generative AI tools.
WebClasSeg-25: A Dual-Classified Webpage Segmentation Dataset - Integrating Functional and Maturity-Based Analysis
Authors: Jonathan Gerber, Jasmin S. Saxer, Kimia Rabishokr, Bruno B. Kreiner, Andreas Weiler
Venue: SIGIR
Year: 2025

Webpage segmentation is a crucial task in web analysis, enabling improvements in information retrieval, user experience, and automated web understanding. However, existing segmentation datasets often lack both comprehensive visual and textual segmentation, as well as classification systems that capture the functional and qualitative aspects of webpages. In this paper, we introduce a novel webpage segmentation dataset that addresses these gaps by providing both visual and textual segmentations, alongside two classification frameworks. The first framework defines a nominal classification of segments based on their functional roles, such as main content, header, footer, and navigation bar. The second introduces an ordinal classification assessing the digital maturity of webpage segments, offering a structured evaluation of their design evolution and complexity. By integrating both classification schemes into a single dataset, our approach enables a more holistic analysis of webpage structures. Furthermore, given the rapid evolution of web design conventions, content structures, and technological trends, our dataset is designed to reflect contemporary webpage characteristics, ensuring its relevance for modern applications in web analysis and machine learning. Additionally, we provide first results on visual segmentation, demonstrating the effectiveness of our dataset in practical applications.
Advancing Scientific Knowledge Retrieval and Reuse with a Novel Digital Library for Machine-Readable Knowledge
Authors: Hadi Ghaemi, Lauren Snyder, Markus Stocker
Venue: SIGIR
Year: 2025

Digital libraries for research, such as the ACM Digital Library or Semantic Scholar, do not enable the machine-supported, efficient reuse of scientific knowledge (e.g., in synthesis research). This is because these libraries are based on document-centric models with narrative text knowledge expressions that require manual or semi-automated knowledge extraction, structuring, and organization. We present ORKG reborn, an emerging digital library that supports finding, accessing, and reusing accurate, fine-grained, and reproducible machine-readable expressions of scientific knowledge that relate scientific statements and their supporting evidence in terms of data and code. The rich expressions of scientific knowledge are published as reborn (born-reusable) articles and provide novel possibilities for scientific knowledge retrieval, for instance by statistical methods, software packages, variables, or data matching specific constraints. We describe the proposed system and demonstrate its practical viability and potential for information retrieval in contrast to state-of-the-art digital libraries and document-centric scholarly communication using several published articles in research fields ranging from computer science to soil science. Our work underscores the enormous potential of scientific knowledge databases and a viable approach to their construction.
The Viability of Crowdsourcing for RAG Evaluation
Authors: Lukas Gienapp, Tim Hagen, Maik Fröbe, Matthias Hagen, Benno Stein, Martin Potthast, Harrisen Scells
Venue: SIGIR
Year: 2025

How good are humans at writing and judging responses in retrieval-augmented generation (RAG) scenarios? To answer this question, we investigate the efficacy of crowdsourcing for RAG through two complementary studies: response writing and response utility judgment. Our new Webis Crowd RAG Corpus 2025 (Webis-CrowdRAG-25) consists of 903 human-written and 903 LLM-generated responses for the 301 topics of the TREC 2024 RAG~track, with each response composed according to one of the three discourse styles 'bullet list', 'essay', or 'news'. For a selection of 65 topics, the corpus further contains 47,320 pairwise human judgments and 10,556 pairwise LLM judgments across seven utility dimensions (e.g., coverage and coherence). Our analyses give insights into human writing behavior for RAG and the viability of crowdsourcing for RAG evaluation. We find that human pairwise judgments provide reliable and cost-effective results. This is much less the case for LLM-based pairwise and human/LLM-based pointwise judgments, nor for automated comparisons with human-written reference responses. All our data and tools are freely available.
From Keywords to Concepts: A Late Interaction Approach to Semantic Product Search on IKEA.com
Authors: Amritpal Singh Gill, Sannikumar Patel, Péter Varga, Patrick Miller, Sakis Athanasiadis
Venue: SIGIR
Year: 2025

Modern e-commerce platforms require search engines that go beyond simple keyword matching to accurately capture customer intent. Traditional keyword-based retrieval struggles with complex, multi-attribute queries, potentially leading to suboptimal results and poor customer experience. To address these challenges, we introduce a late interaction-based semantic search engine designed for IKEA product search. This approach significantly improves retrieval quality while maintaining low latency, ensuring a more effective and seamless search experience for customers. Our approach departs from single-vector embeddings by leveraging token-level late interaction scoring, enabling fine-grained alignment between search queries and product descriptions. To enhance search effectiveness, we introduce three key contributions: (1) large-scale synthetic query generation using LLMs to diversify training data (2) strong negative sampling to improve contrastive learning, and (3) adaptive thresholding to dynamically refine ranking cutoffs and prevent over-retrieval biases. In live A/B testing on IKEA.com for long tail queries in the U.S. market, our system outperforms IKEA Boolean search with a 3.1% increase in click-through rate, a 1.96% boost in conversions, 1.78% increase in search interaction rate, and a 2.18% rise in add-to-cart actions. These results validate the effectiveness of efficient token-level retrieval and adaptive ranking in large-scale commercial search.
Refined Medical Search via Dense Retrieval and User Interaction
Authors: Reyhaneh Goli, Alistair Moffat, George Buchanan
Venue: SIGIR
Year: 2025

Users formulate search queries that reflect an information need. Those queries are then submitted to a search service in the expectation that the retrieved results will allow the user to complete an external task, and align with their broader information context. In this study, we reimplement and reproduce the log-augmented dense retrieval approach introduced by Jin, Shin, and Lu in 2023. As part of our study we extend the experimentation by: (1) using all of the available training data rather than a subset; (2) exploring a second dense retrieval model; and (3) enhancing the approach by incorporating user reformulation behavior into the dense retrieval computation so as to improve ranking effectiveness. To evaluate these modifications, we again utilize the TripClick IR benchmark, which comprises approximately four million click log entries from a health domain web search engine. Although we were unable to exactly replicate the results of Jin, Shin, and Lu, our findings confirm the overall trends reported in their study. Specifically, our results support the conclusion that incorporating user interactions into dense retrieval models improves ranking effectiveness compared to when no user information is available. Moreover, our enhanced formulation allows further small gains to be made in retrieval effectiveness.
Measuring Text-Image Retrieval Fairness with Synthetic Data
Authors: Lluís Gómez
Venue: SIGIR
Year: 2025

In this paper, we study social bias in cross-modal text-image retrieval systems, focusing on the interaction between textual queries and image responses. Despite the significant advancements in cross-modal retrieval models, the potential for social bias in their responses remains a pressing concern, necessitating a comprehensive framework for assessment and mitigation. We introduce a novel framework for evaluating social bias in cross-modal retrieval systems, leveraging a new dataset and appropriate metrics specifically designed for this purpose. Our dataset, Social Inclusive Synthetic Professionals Images (SISPI), comprises 49K images generated using state-of-the-art text-to-image models, ensuring a balanced representation of demographic groups across various professional roles. We use this dataset to conduct an extensive analysis of social bias (gender and ethnic) in state of the art cross-modal retrieval deep models, including CLIP, ALIGN, BLIP, FLAVA, COCA, and many others. Using diversity metrics, grounded in the distribution of different demographic groups' images in the retrieval rankings, we provide a quantitative measure of fairness, facilitating a detailed analysis of models' behavior. Our work sheds light on biases present in current cross-modal retrieval systems and emphasizes the importance of training data curation, providing a foundation for future research and development towards more equitable and unbiased models. The dataset and code of our framework is publicly available at https://sispi-benchmark.github.io/sispi-benchmark/.
Pyramid Mixer: Multi-dimensional Multi-period Interest Modeling for Sequential Recommendation
Authors: Zhen Gong, Zhifang Fan, Hui Lu, Qiwei Chen, Chenbin Zhang, Lin Guan, Yuchao Zheng, Feng Zhang, Xiao Yang, Zuotao Liu
Venue: SIGIR
Year: 2025

Sequential recommendation, a critical task in recommendation systems, predicts the next user action based on the understanding of the user's historical behaviors. Conventional studies mainly focus on cross-behavior modeling with self-attention based methods while neglecting comprehensive user interest modeling for more dimensions. In this study, we propose a novel sequential recommendation model, Pyramid Mixer, which leverages the MLP-Mixer architecture to achieve efficient and complete modeling of user interests. Our method learns comprehensive user interests via cross-behavior and cross-feature user sequence modeling. The mixer layers are stacked in a pyramid way for cross-period user temporal interest learning. Through extensive offline and online experiments, we demonstrate the effectiveness and efficiency of our method, and we obtain a +0.106% improvement in user stay duration and a +0.0113% increase in user active days in the online A/B test. The Pyramid Mixer has been successfully deployed on the industrial platform, demonstrating its scalability and impact in real-world applications.
Fast and Effective Early Termination for Simple Ranking Functions
Authors: Jinrui Gou, Antonio Mallia, Yifan Liu, Minghao Shao, Torsten Suel
Venue: SIGIR
Year: 2025

Web search engines often perform an initial candidate generation phase using a fast and simple ranking function, followed by subsequent reranking with more expensive rankers. Such simple ranking functions usually compute the score of a document as the sum of term-wise impact scores, and they include traditional baselines such as BM25 and Query Likelihood, as well as some recently proposed learned sparse models based on document expansion and learned impact scores. In this paper, we explore extremely fast and highly effective early termination techniques for such simple ranking functions. Our extensive experiments with a number of different ranking functions show that our methods achieve very fast response times on MSMarco V1 and V2 data while maintaining retrieval quality close to that of a safe and much slower baseline.
An EEG Dataset of Word-level Brain Responses for Semantic Text Relevance
Authors: Vadym Gryshchuk, Michiel M. A. Spapé, Maria Maistro, Christina Lioma, Tuukka Ruotsalo
Venue: SIGIR
Year: 2025

Electroencephalography (EEG) can enable non-invasive, real-time measurement of brain activity reflecting cognitive processes during human language processing. Previously released EEG datasets primarily capture brain signals recorded either during natural reading or within controlled psycholinguistic experimental settings. Given that information retrieval research depends on understanding and modelling relevance, we present a novel dataset including EEG data recorded while participants read text that is semantically relevant or irrelevant to self-selected topics. The dataset contains 23, 270 time-locked (∼ 0.7s) word-level EEG recordings. Using these data, we conduct benchmark experiments with two evaluation protocols, cross-subject and within-subject, focusing on two prediction tasks: word relevance and sentence relevance. We report the performance of five well known models on these tasks. Altogether, our dataset paves the way for advancing research on language relevance, brain input and feedback-based recommendation and retrieval systems, and development of brain-computer interface (BCI) devices for online detection of language relevance. Our dataset and code are openly released at https://osf.io/xh3g5/wiki/home/ and at HuggingFace https://huggingface.co/datasets/Quoron/EEG-semantic-text-relevance.
Enhancing New-item Fairness in Dynamic Recommender Systems
Authors: Huizhong Guo, Zhu Sun, Dongxia Wang, Tianjun Wei, Jinfeng Li, Jie Zhang
Venue: SIGIR
Year: 2025

New-items play a crucial role in recommender systems (RSs) for delivering fresh and engaging user experiences. However, traditional methods struggle to effectively recommend new-items due to their short exposure time and limited interaction records, especially in dynamic recommender systems (DRSs) where new-items get continuously introduced and users' preferences evolve over time. This leads to significant unfairness towards new-items, which could accumulate over the successive model updates, ultimately compromising the stability of the entire system. Therefore, we propose FairAgent, a reinforcement learning (RL)-based new-item fairness enhancement framework specifically designed for DRSs. It leverages knowledge distillation to extract collaborative signals from traditional models, retaining strong recommendation capabilities for old-items. In addition, FairAgent introduces a novel reward mechanism for recommendation tailored to the characteristics of DRSs, which consists of three components: 1) a new-item exploration reward to promote the exposure of dynamically introduced new-items, 2) a fairness reward to adapt to users' personalized fairness requirements for new-items, and 3) an accuracy reward which leverages users' dynamic feedback to enhance recommendation accuracy. Extensive experiments on three public datasets and backbone models demonstrate the superior performance of FairAgent. The results present that FairAgent can effectively boost new-item exposure, achieve personalized new-item fairness, while maintaining high recommendation accuracy.
Hyperbolic Multi-Criteria Rating Recommendation
Authors: Zhihao Guo, Ting Han, Peng Song, Chenjiao Feng, Kaixuan Yao, Jiye Liang
Venue: SIGIR
Year: 2025

Multi-criteria (MC) ratings as auxiliary supervisory signals can improve the prediction accuracy of recommender systems. The existing MC methods learn the representations of users and items in Euclidean space to estimate the interaction probabilities. However, this modeling paradigm ignores two important aspects. Firstly, when embedding power-law distribution data and personalized MC preferences in Euclidean space, the model may produce suboptimal solutions due to the distortion of the hierarchical structure. Secondly, the inevitable noise in MC ratings may hinder the recommendation quality of the model. To address the above issues, we propose a novel framework called Hyperbolic Multi-Criteria Recommendation (HMCR), which aims to mine users' MC behavioral features on hyperbolic manifolds and mitigate the noise interference through knowledge transfer among the criteria. Specifically, we map the representations on each criterion view to a hyperbolic space with adjustable curvature based on the Lorentz model, which is used to capture the hierarchical structure of collective user behavior. The MC preferences of individual users are fused by calculating the hyperbolic attention among each criterion and the overall rating. Moreover, we design a self-supervised contrastive loss to suppress the negative impact of noise interactions on the model. The experimental results on four real-world datasets show that the HMCR significantly outperforms the existing baselines.
Retrieval for Semantic People Search
Authors: Rupesh Gupta, Chujie Zheng, Haojun Li
Venue: SIGIR
Year: 2025

These days we have a large number of open-source embedding LLMs that can be leveraged in a bi-encoder architecture for retrieval. However, they do not perform very well when leveraged as is for people search because the (query, document) pairs encountered in people search are much more complex than the text pairs that these open-source embedding LLMs are trained on. In this paper we present our experiments, learnings and solution to the problem of retrieval for people search. Our solution involves query simplification, document simplification, fine-tuning of a 7B parameter embedding LLM, and compression of embeddings through Matryoshka learning.