OpenRaiser/Intern-Atlas huggingface dataset:
Investigating Task Arithmetic for Zero-Shot Information Retrieval
Authors: Marco Braga, Pranav Kasela, Alessandro Raganato, Gabriella Pasi
Venue: SIGIR
Year: 2025

Large Language Models (LLMs) have shown impressive zero-shot performance across a variety of Natural Language Processing tasks, including document re-ranking. However, their effectiveness degrades on unseen tasks and domains, largely due to shifts in vocabulary and word distributions. In this paper, we investigate Task Arithmetic, a technique that combines the weights of LLMs pre-trained on different tasks or domains via simple mathematical operations, such as addition or subtraction, to adapt retrieval models without requiring additional fine-tuning. Our method is able to synthesize diverse tasks and domain knowledge into a single model, enabling effective zero-shot adaptation in different retrieval contexts. Extensive experiments on publicly available scientific, biomedical, and multilingual datasets show that our method improves state-of-the-art re-ranking performance by up to 18% in NDCG@10 and 15% in P@10. In addition to these empirical gains, our analysis provides insights into the strengths and limitations of Task Arithmetic as a practical strategy for zero-shot learning and model adaptation. We make our code publicly available at https://github.com/DetectiveMB/Task-Arithmetic-for-ZS-IR.
IR for AAC Users: A Hyperdimensional Computing (Vector Symbolic Architectures) Approach
Authors: Hunter Briegel, Maya Pagal, J. Shane Culpepper
Venue: SIGIR
Year: 2025

This work proposes Hyperdimensional Computing (HDC) as a design paradigm [13] to facilitate search and recommendation activities for disabled users employing symbolic augmentative and alternative communication (AAC) systems. Such a context necessitates flexibility and composability in item and query representations as a consequence of vocabularies being tailored to an individual user. HDC is suggested to meet these needs in an efficient manner. However, construction and empirical evaluation are left to additional research. Project development for symbolic AAC supports a small but highly diverse user base. This creates a ''low resource environment'' from both an economic and technical perspective. The heterogeneity of user communication abilities, preferences, and symbolic interpretation limits the generalisability of datasets and presents a cold start problem. Furthermore, countries with underdeveloped social welfare and health infrastructure, the Philippines serving as an illustrative example, have few practitioners able to fine-tune AAC devices or support inclusive decision making. Consequently, this exacerbates resource constraints, which motivates systems to offer consumer-level ease of use. Rural areas, in any socioeconomic setting, lack reliable network connectivity: designers should anticipate regular on-device computation, leaving networked tasks to limited occasions. Technologies for disabilities, broadly speaking, are related to all 17 UN Sustainable Development Goals [21]. Additionally, AAC is a focus area of UNICEF's work on disabilities and inclusion for marginalised children [2]. By proposing HDC as a way to mediate diverse feature spaces, search and recommendation applications are able to reuse semantic components and process structures - reducing costs. This flexibility enables adaptation to underserved cultures, where localised symbol interpretations are particularly needed [3]. A similar approach has been taken to circulate open access symbols [2]. HDC's development comes from an interdisciplinary perspective, implementing cognitive and linguistic science precepts computationally [8]. Building on this idea, AAC for search requires wider collaboration with entities outside the traditional information retrieval (IR) community, such as speech pathologists, disability support workers, and end users with disabilities.
Efficient Re-ranking with Cross-encoders via Early Exit
Authors: Francesco Busolin, Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, Salvatore Trani, Alberto Veneri
Venue: SIGIR
Year: 2025

Pre-trained language models based on transformer networks are highly effective for document re-ranking in ad-hoc search. Among these, cross-encoders stand out for their effectiveness, as they process query-document pairs through the entire transformer network to compute ranking scores. However, this traversal is computationally expensive. To address this, prior work has explored early-exit strategies, enabling the model to terminate the traversal of query-document pairs. These techniques rely on learned classifiers, placed after each transformer block, that decide if a query-document pair can be dropped. Diverging from previous approaches, we propose Similarity-based Early Exit (SEE), a novel-non-learned-strategy that exploits the similarities between query and document token embeddings to early-terminate the inference of documents that will most likely be non-relevant to the query. Even though SEE can be used after every transformer block, we show that the best advantage is achieved when applied before the first transformer block, thus saving most of the inference cost for the query-document pairs. Reproducible experiments on 17 public datasets covering in-domain and out-of-domain evaluation show that SEE can be effectively applied to four different cross-encoders, achieving speedups of up to 3.5× with a limited loss in ranking effectiveness.
A Knowledge Extraction Framework on Cyber Threat Reports with Enhanced Security Profiles
Authors: Yongxin Cai, Jing Qiu, Fan Zhang, Qiang Li, Lei Chen
Venue: SIGIR
Year: 2025

Knowledge extraction on Cyber Threat Reports (CTRs) is critical for attack investigation and defenses. The granularity and the usability of the knowledge are key issues: the former is determined by entity recognition on CTRs, whereas the latter mainly depends on proper relation extraction. Nevertheless, in the state-of-the-art entity recognition methods on CTRs using span representation, the local semantics of behavior are not considered and the sequential features of entity labels within behavior descriptions are not utilized. Besides, domain-specific definitions/forms of the relation types and knowledge representations are also crucial for effective utilization of knowledge. In this paper, we propose a novel knowledge extraction framework on CTRs to address the above concerns. The framework is formed by the Enhanced Security Profiles (ESP) that can be directly utilized by security detection devices. In the ESP framework, we propose 3 modules to facilitate fine-grained and accurate knowledge extractions: (1) The entity recognition module utilizes a label-aware subsequence autoregressive algorithm to integrate local semantic and label sequence features, enabling accurate identification of cybersecurity entities; (2) The relation extraction module employs LLM-based strategies with shared partition representations to enhance semantic understanding and domain relevance; and (3) The security profile generation module leverages Chain-of-Thought reasoning and In-Context Learning to produce machine-readable rules executable in security detection systems. Extensive experiments on 6 datasets demonstrate that the ESP framework largely outperform the state-of-the-art solutions e.g., the Micro-Fl scores on entity recognition and relation extraction are at least 1.54% and 13.12% better, respectively. Our code can be found in https://github.com/YxinMiracle/ESP.
Interest Changes: Considering User Interest Life Cycle in Recommendation System
Authors: Yinjiang Cai, Jiangpan Hou, Yangping Zhu, Yuan Nie
Venue: SIGIR
Year: 2025

In recommendation systems, user interests are always in a state of constant flux. Typically, a user interest experiences a emergent phase, a stable phase, and a declining phase, which are referred to as the ''user interest life-cycle''. Recent papers on user interest modeling have primarily focused on how to compute the correlation between the target item and user's historical behaviors, without thoroughly considering the life-cycle features of user interest. In this paper, we propose an effective method called Deep Interest Life-cycle Network (DILN), which not only captures the interest life-cycle features efficiently, but can also be easily integrated to existing ranking models. DILN contains two key components: Interest Life-cycle Encoder Module constructs historical activity histograms of the user interest and then encodes them into dense representation. Interest Life-cycle Fusion Module injects the encoded dense representation into multiple expert networks, with the aim of enabling the specific phase of interest life-cycle to activate distinct experts. Online A/B testing reveals that DILN achieves significant improvements of +0.38% in CTR, +1.04% in CVR and +0.25% in duration per user, which demonstrates its effectiveness. In addition, DILN inherently increase the exposure of users' emergent and stable interests while decreasing the exposure of declining interests. DILN has been deployed on the Lofter App.
Exploring Training and Inference Scaling Laws in Generative Retrieval
Authors: Hongru Cai, Yongqi Li, Ruifeng Yuan, Wenjie Wang, Zhen Zhang, Wenjie Li, Tat-Seng Chua
Venue: SIGIR
Year: 2025

Generative retrieval reformulates retrieval as an autoregressive generation task, where large language models (LLMs) generate target documents directly from a query. As a novel paradigm, the mechanisms that underpin its performance and scalability remain largely unexplored. We systematically investigate training and inference scaling laws in generative retrieval, exploring how model size, training data scale, and inference-time compute jointly influence performance. We propose a novel evaluation metric inspired by contrastive entropy and generation loss, providing a continuous performance signal that enables robust comparisons across diverse generative retrieval methods. Our experiments show that n-gram-based methods align strongly with training and inference scaling laws. We find that increasing model size, training data scale, and inference-time compute all contribute to improved performance, highlighting the complementary roles of these factors in enhancing generative retrieval. Across these settings, LLaMA models consistently outperform T5 models, suggesting a particular advantage for larger decoder-only models in generative retrieval. Our findings underscore that model sizes, data availability, and inference computation interact to unlock the full potential of generative retrieval, offering new insights for designing and optimizing future systems. We release code at SLGR GitHub repository.
Agentic Feedback Loop Modeling Improves Recommendation and User Simulation
Authors: Shihao Cai, Jizhi Zhang, Keqin Bao, Chongming Gao, Qifan Wang, Fuli Feng, Xiangnan He
Venue: SIGIR
Year: 2025

Large language model-based agents are increasingly applied in the recommendation field due to their extensive knowledge and strong planning capabilities. While prior research has primarily focused on enhancing either the recommendation agent or the user agent individually, the collaborative interaction between the two has often been overlooked. Towards this research gap, we propose a novel framework that emphasizes the feedback loop process to facilitate the collaboration between the recommendation agent and the user agent. Specifically, the recommendation agent refines its understanding of user preferences by analyzing the feedback from the user agent on the item recommendation. Conversely, the user agent further identifies potential user interests based on the items and recommendation reasons provided by the recommendation agent. This iterative process enhances the ability of both agents to infer user behaviors, enabling more effective item recommendations and more accurate user simulations. Extensive experiments on three datasets demonstrate the effectiveness of the agentic feedback loop: the agentic feedback loop yields an average improvement of 11.52% over the single recommendation agent and 21.12% over the single user agent. Furthermore, the results show that the agentic feedback loop does not exacerbate popularity or position bias, which are typically amplified by the real-world feedback loop, highlighting its robustness. The source code is available at https://github.com/Lanyu0303/AFL.
Unveiling DIME: Reproducibility, Generalizability, and Formal Analysis of Dimension Importance Estimation for Dense Retrieval
Authors: Cesare Campagnano, Antonio Mallia, Fabrizio Silvestri
Venue: SIGIR
Year: 2025

Dimension IMportance Estimation (DIME) is a recently proposed technique to enhance ranking effectiveness of dense retrieval models by pruning irrelevant embedding dimensions through Pseudo Relevance Feedback (PRF DIME) or exploiting dense representations of Large Language Model-generated answers (LLM DIME). Despite strong empirical performance, its theoretical foundations and generalizability remain open questions. In this paper, we propose four key contributions. First, we provide a rigorous theoretical analysis of DIME, framing it as a denoising mechanism that mitigates embedding noise while preserving the salient information. Second, we conduct a comprehensive reproducibility study, confirming previously reported gains for both PRF DIME and LLM DIME. Third, we extend the evaluations of PRF DIME by applying it to a broader set of embedding models with distinct characteristics, such as matryoshka embeddings, cosine similarity-optimized models, and architectures that produce high-dimensional representations, while also testing it on diverse retrieval datasets. For LLM DIME, we expand the analysis across a range of LLMs, comparing high-parameter proprietary models with cheaper open-source alternatives. Finally, we refine DIME by introducing an attention-inspired PRF mechanism and propose to leverage dimension importance as a reranking technique.
Dynamic Superblock Pruning for Fast Learned Sparse Retrieval
Authors: Parker Carlson, Wentai Xie, Shanxiu He, Tao Yang
Venue: SIGIR
Year: 2025

This paper proposes superblock pruning (SP) during top-k online document retrieval for learned sparse representations. SP structures the sparse index as a set of superblocks on a sequence of document blocks and conducts a superblock-level selection to decide if some superblocks can be pruned before visiting their child blocks. SP generalizes the previous flat block or cluster-based pruning, allowing the early detection of groups of documents that cannot or are less likely to appear in the final top-k list. SP can accelerate sparse retrieval in a rank-safe or approximate manner under a high-relevance competitiveness constraint. Our experiments show that the proposed scheme significantly outperforms state-of-the-art baselines on MS MARCO passages on a single-threaded CPU.
The LiveRAG Challenge at SIGIR 2025
Authors: David Carmel, Simone Filice, Guy Horowitz, Yoelle Maarek, Oren Somekh, Ran Tavory
Venue: SIGIR
Year: 2025

The LiveRAG Challenge at SIGIR 2025 provides a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants from academia and industry have been invited to build a RAG-based question answering system using a fixed corpus (Fineweb-10BT) and a common open-source LLM (Falcon3-10B-Instruct). The goal is to enable fair, focused comparisons on retrieval and prompting strategies. During the Live Challenge Day, the competing teams must provide answers and supportive information to 500 unseen questions within a strict two-hour window. Evaluation is conducted in two stages: automated LLM-as-a-judge scoring mechanism for correctness and faithfulness, followed by a manual review of top ranked submissions. The winners will be announced and prizes awarded during the LiveRAG Workshop at SIGIR 2025 in Padua, Italy.
Generating Effective Health-Related Queries for Promoting Reliable Search Results
Authors: Xiana Carrera, Marcos Fernández-Pichel, David E. Losada
Venue: SIGIR
Year: 2025

Misinformation on the Internet poses significant risks to users seeking health information. This paper addresses the challenge of generating effective health-related queries to promote reliable search results. We propose a method leveraging Large Language Models to generate synthetic narratives that guide the creation of alternative queries. These queries are designed to retrieve more helpful and fewer harmful documents compared to those retrieved by the original user queries. We evaluate the effectiveness of these queries using classic and neural retrieval models across multiple datasets, demonstrating promising improvements in retrieving reputable content.
Adaptive Domain Scaling for Personalized Sequential Modeling in Recommenders
Authors: Zheng Chai, Hui Lu, Di Chen, Qin Ren, Yuchao Zheng, Xun Zhou
Venue: SIGIR
Year: 2025

Users generally exhibit complex behavioral patterns and diverse intentions in multiple business scenarios of super APPs, presenting great challenges to industrial multi-domain recommenders. Current researches and practices generally emphasize sophisticated network structures to accommodate diverse data distributions, while neglecting the inherent understanding of user behavioral sequence from the multi-domain perspective. In this paper, we present Adaptive Domain Scaling (ADS) model, which comprehensively enhances the personalization capability in target-aware sequence modeling across multiple domains. Specifically, ADS comprises of two major modules, including personalized sequence representation generation (PSRG) and personalized candidate representation generation (PCRG). The modules contribute to the tailored multi-domain modeling by dynamically learning both the user interacted item representation and the candidate target item representation, facilitating adaptive user intention understanding. Experiments on both a public and two billion-scaled industrial datasets, and online A/B tests on two influential business scenarios at ByteDance validate its effectiveness. Currently, ADS has been fully deployed in dozens of recommendation services at ByteDance, serving billions of users.
Inside Out 2: Make Room for New Emotions & LLM: A Reproducibility Study of the Emotional Side of Search in the Classroom
Authors: Hrishita Chakrabarti, Diletta Micol Tobia, Monica Landoni, Maria Soledad Pera
Venue: SIGIR
Year: 2025

In an existing study, the InsideOut Framework is used to produce and explore the emotional profiles of search engines (SE) in response to queries formulated by children aged 9 to 11 in the classroom context, revealing the emotional diversity of SE responses. Since then, there have been significant technological advances in emotion detection and information access. In this work, we conduct a comprehensive reproducibility study where we probe today's emotional profile of SE using both a lexicon-based and a language-model based approach tailored to the Italian language, thus addressing an acknowledged limitation of the original study. Additionally, considering the prevalence of agents based on Large Language Models (LLM) as information access systems among children, we extend the analysis to capture the emotional undertones of LLM responses and juxtapose them to those of SE. Our findings emphasize the importance of leveraging the appropriate emotion detection technique to produce and explore emotional profiles and lead us to reflect on the interplay of emotions on children's search-as-learning experience.
General Neural Embedding for Sequence Distance Approximation
Authors: Zhihao Chang, Ding Wang, Xiu Tang, Kingsum Chow, Jianwei Yin
Venue: SIGIR
Year: 2025

Sequence distance computation is a critical and fundamental task in many fields, such as bioinformatics, and time series analysis. Traditional functions for computing the distance between sequences are often based on dynamic programming to find a globally optimal alignment, which has quadratic complexity and is difficult to parallelize, thus limiting their application in large-scale datasets with long sequences. To solve this problem, various fields have designed some specialized models to approximate these distance functions inspired by deep representation learning, i.e., projecting the sequence into a geometric embedding space through an embedding function, so that the distance between sequences can be approximated by the distance in the high-dimensional embedding space, thereby reducing the quadratic complexity to linear. However, we note that even though the element types in sequence and distance functions are different across various fields, the core problem that needs to be solved remains the same. In this paper, we attempt to unify the sequence distance computation approximation from various fields and propose GnesDA. Specifically, we first unify the input representation of sequences in which the element type is the symbol and numeric values. We then encode the sequence using a convolutional block and a Transformer block sequentially, which can effectively capture local patterns and long dependencies respectively. Extensive experiments on four distance functions as well as four large-scale real-world datasets demonstrate that GnesDA achieves state-of-the-art in terms of both versatility and effectiveness. For the task of similarity retrieval, GnesDA can improve the edit distance, NW distance, DTW, and EDR by an average of 10.55%, 6.67%, 4.51%, and 12.00% on all metrics.
Lost in Transliteration: Bridging the Script Gap in Neural IR
Authors: Andreas Chari, Iadh Ounis, Sean MacAvaney
Venue: SIGIR
Year: 2025

Most human languages use scripts other than the Latin alphabet. Search users in these languages often formulate their information needs in a transliterated -usually Latinized- form for ease of typing. For example, Greek speakers might use Greeklish, and Arabic speakers might use Arabizi. This paper shows that current search systems, including those that use multilingual dense embeddings such as BGE-M3, do not generalise to this setting, and their performance rapidly deteriorates when exposed to transliterated queries. This creates a ''script gap'' between the performance of the same queries when written in their native or transliterated form. We explore whether adapting the popular ''translate-train'' paradigm to transliterations can enhance the robustness of multilingual Information Retrieval (IR) methods and bridge the gap between native and transliterated scripts. https://github.com/andreaschari/transliterations By exploring various combinations of non-Latin and Latinized query text for training, we investigate whether we can enhance the capacity of existing neural retrieval techniques and enable them to apply to this important setting. We show that by further fine-tuning IR models on an even mixture of native and Latinized text, they can perform this cross-script matching at nearly the same performance as when the query was formulated in the native script. Out-of-domain evaluation and further qualitative analysis show that transliterations can also cause queries to lose some of their nuances, motivating further research.