OpenRaiser/Intern-Atlas huggingface dataset:
Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks
Authors: Antonios Tragoudaras, Theofanis Aslanidis, Emmanouil Georgios Lionis, Marina Orozco González, Panagiotis Eustratiadis
Venue: SIGIR
Year: 2025

Text data are often encoded as dense vectors, known as embeddings, which capture semantic, syntactic, contextual, and domain-specific information. These embeddings, widely adopted in various applications, inherently contain rich information that may be susceptible to leakage under certain attacks. The GEIA framework highlights vulnerabilities in sentence embeddings, demonstrating that they can reveal the original sentences they represent. In this study, we reproduce GEIA's findings across various neural sentence embedding models. Additionally, we contribute new analysis to examine whether these models leak sensitive information from their training datasets. We propose a simple yet effective method without any modification to the attacker's architecture proposed in GEIA. The key idea is to examine differences between log-likelihood for masked and original variants of data that sentence embedding models have been pre-trained on, calculated on the embedding space of the attacker. Our findings indicate that following our approach, an adversary party can recover meaningful sensitive information related to the pre-training knowledge of the popular models used for creating sentence embeddings, seriously undermining their security. Our code is available on: https://github.com/taslanidis/GEIA
Robust Fine-tuning for Retrieval Augmented Generation against Retrieval Defects
Authors: Yiteng Tu, Weihang Su, Yujia Zhou, Yiqun Liu, Qingyao Ai
Venue: SIGIR
Year: 2025

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieved from a knowledge base. However, its effectiveness is fundamentally constrained by the reliability of both the retriever and the knowledge base (i.e., the retrieval system). In real-world scenarios, imperfections in these components often lead to the retrieval of noisy, irrelevant, or misleading counterfactual information, ultimately undermining the trustworthiness of RAG systems. To address this challenge, we propose Robust Fine-Tuning (RbFT), a method designed to enhance the resilience of LLMs against retrieval defects through two targeted fine-tuning tasks. Experimental results demonstrate that RbFT significantly improves the robustness of RAG systems across diverse retrieval conditions, surpassing existing methods while maintaining high inference efficiency and compatibility with other robustness techniques.
Meta-Learning for Incomplete Multimodal Sentiment Analysis
Authors: Geng Tu, Tianhao Wu, Xuan Luo, Xi Zeng, Wenjie Li, Ruifeng Xu
Venue: SIGIR
Year: 2025

Modality incompleteness is a critical yet underexplored challenge in multimodal sentiment analysis (MSA). Existing efforts, trained and evaluated under fixed missing rates, struggle to adapt to real-world scenarios with varying missing rates. To address this, we propose the Missing Modality Adaptation Framework (M2AF), leveraging model-agnostic meta-learning to enhance robustness against different levels of modality incompleteness. M2AF operates in two stages: meta-training and meta-testing. In the meta-training stage, a pre-trained MSA model, initially optimized for fixed missing rates, is further adapted to different levels of missing rates-low, moderate, and high. In the meta-testing stage, the model rapidly updates its parameters using minimal training data to handle target missing scenarios. Experiments on two popular datasets demonstrate that M2AF significantly improves the performance and generalization of various MSA models, ensuring more robust sentiment analysis in real-world settings with different modality incompleteness.
From Monolith to Mosaic: Uncovering Behavioral Differences for Choice Models in Recommender Systems Simulations
Authors: Robin Ungruh, Alejandro Bellogín, Maria Soledad Pera
Venue: SIGIR
Year: 2025

Simulation is widely used in recommender systems research to study algorithm behavior and its impact on users. A common strategy involves adopting a universal choice model to represent users, assuming all follow the same consumption patterns. This one-size-fits-all approach overlooks the diversity in user preferences and decision-making patterns. In this work, we scrutinize whether this universal view fails to account for unique user behavior, thus harming realism and reliability of simulation outcomes. We conduct multiple simulations with various recommendation algorithms and choice models in the movie domain, comparing outcomes to users' organic consumption patterns. Further, we evaluate whether a holistic model that captures users' differences in behavior would better reflect a wide user base. Our findings highlight the limitations of using a naive, universal choice model and emphasize the need for more nuanced, user-specific approaches to make contributions from simulation studies more reflective of real-world effects.
Inquiry Assistant Using LLM-Generated Knowledge Graphs
Authors: István Varga, Yuta Yamashita
Venue: SIGIR
Year: 2025

Businesses are increasingly overwhelmed by inquiries related to their services or products. Relying on human agents to handle inquiries via email results in higher costs and delayed responses, contributing to customer dissatisfaction. In response to these challenges, this pilot study leverages advancements in Large Language Models (LLMs) by proposing a fully automated method for generating a knowledge graph from unstructured data in help pages, which is then utilized to power a fully automated dialogue management system. By transitioning to a chat-based approach, our method aims to handle ambiguous, incomplete, or nonspecific inquiries more effectively and enhance customer satisfaction with tailored, natural responses. We also implement explicit safeguards to improve intent identification and prevent response hallucinations. We validate our proposal in the hotel industry, demonstrating that our knowledge graph based AI agent outperforms the baseline Retrieval-Augmented Generation (RAG) model in accuracy while facilitating more natural and coherent dialogues.
Resource for Error Analysis in Text Simplification: New Taxonomy and Test Collection
Authors: Benjamin Vendeville, Liana Ermakova, Pierre De Loor
Venue: SIGIR
Year: 2025

The general public often encounters complex texts but does not have the time or expertise to fully understand them, leading to the spread of misinformation. Automatic Text Simplification (ATS) helps make information more accessible, but its evaluation methods have not kept up with advances in text generation, especially with Large Language Models (LLMs). In particular, recent studies have shown that current ATS metrics do not correlate with the presence of errors. Manual inspections have further revealed a variety of errors, underscoring the need for a more nuanced evaluation framework, which is currently lacking. This resource paper addresses this gap by introducing a test collection for detecting and classifying errors in simplified texts. First, we propose a taxonomy of errors, with a formal focus on information distortion. Next, we introduce a parallel dataset of automatically simplified scientific texts. This dataset has been human-annotated with labels based on our proposed taxonomy. Finally, we analyze the quality of the dataset, and we study the performance of existing models to detect and classify errors from that taxonomy. These contributions give researchers the tools to better evaluate errors in ATS, develop more reliable models, and ultimately improve the quality of automatically simplified texts.
AdSight: Scalable and Accurate Quantification of User Attention in Multi-Slot Sponsored Search
Authors: Mario Villaizán-Vallelado, Matteo Salvatori, Kayhan Latifzadeh, Antonio Penta, Luis A. Leiva, Ioannis Arapakis
Venue: SIGIR
Year: 2025

Modern Search Engine Results Pages (SERPs) present complex layouts where multiple elements compete for visibility. Attention modelling is crucial for optimising web design and computational advertising, whereas attention metrics can inform ad placement and revenue strategies. We introduce AdSight, a method leveraging mouse cursor trajectories to quantify in a scalable and accurate manner user attention in multi-slot environments like SERPs. AdSight uses a novel Transformer-based sequence-to-sequence architecture where the encoder processes cursor trajectory embeddings, and the decoder incorporates slot-specific features, enabling robust attention prediction across various SERP layouts. We evaluate our approach on two Machine Learning tasks: (1)~regression, to predict fixation times and counts; and (2)~classification, to determine some slot types were noticed. Our findings demonstrate the model's ability to predict attention with unprecedented precision, offering actionable insights for researchers and practitioners.
Intent Representation Learning with Large Language Model for Recommendation
Authors: Yu Wang, Lei Sang, Yi Zhang, Yiwen Zhang
Venue: SIGIR
Year: 2025

Intent-based recommender systems have garnered significant attention for uncovering latent fine-grained preferences. Intents, as underlying factors of interactions, are crucial for improving recommendation interpretability. Most methods define intents as learnable parameters updated alongside interactions. However, existing frameworks often overlook textual information (e.g., user reviews, item descriptions), which is crucial for alleviating the sparsity of interaction intents. Exploring these multimodal intents, especially the inherent differences in representation spaces, poses two key challenges: i) How to align multimodal intents and effectively mitigate noise issues; ii) How to extract and match latent key intents across modalities. To tackle these challenges, we propose a model-agnostic framework, Intent Representation Learning with Large Language Model (IRLLRec), which leverages large language models (LLMs) to construct multimodal intents and enhance recommendations. Specifically, IRLLRec employs a dual-tower architecture to learn multimodal intent representations. Next, we propose pairwise and translation alignment to eliminate inter-modal differences and enhance robustness against noisy input features. Finally, to better match textual and interaction-based intents, we employ momentum distillation to perform teacher-student learning on fused intent representations. Empirical evaluations on three datasets show that our IRLLRec framework outperforms baselines.
DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction
Authors: Kefan Wang, Hao Wang, Wei Guo, Yong Liu, Jianghao Lin, Defu Lian, Enhong Chen
Venue: SIGIR
Year: 2025

Click-through rate (CTR) prediction is a critical task in online advertising and recommender systems, relying on effective modeling of feature interactions. Explicit interactions capture predefined relationships, such as inner products, but often suffer from data sparsity, while implicit interactions excel at learning complex patterns through non-linear transformations but lack inductive biases for efficient low-order modeling. Existing two-stream architectures integrate these paradigms but face challenges such as limited information sharing, gradient imbalance, and difficulty preserving low-order signals in sparse CTR data. We propose a novel framework, DynamicLow-Order-Aware Fusion (DLF), which addresses these limitations through two key components: a Residual-Aware Low-Order Interaction Network (RLI) and a Network-Aware Attention Fusion Module (NAF). RLI explicitly preserves low-order signals while mitigating redundancy from residual connections, and NAF dynamically integrates explicit and implicit representations at each layer, enhancing information sharing and alleviating gradient imbalance. Together, these innovations balance low-order and high-order interactions, improving model expressiveness. Extensive experiments on public datasets demonstrate that DLF achieves state-of-the-art performance in CTR prediction, addressing key limitations of existing models. The implementation is publicly available at https://github.com/USTC-StarTeam/DLF.
InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
Authors: Zixu Wang, Bingbing Xu, Yige Yuan, Huawei Shen, Xueqi Cheng
Venue: SIGIR
Year: 2025

As an important graph pre-training method, Graph Contrastive Learning (GCL) continues to play a crucial role in the ongoing surge of research on graph foundation models or LLM as enhancer for graphs. Traditional GCL optimizes InfoNCE by using augmentations to define self-supervised tasks, treating augmented pairs as positive samples and others as negative. However, this leads to semantically similar pairs being classified as negative, causing significant sampling bias and limiting performance. In this paper, we argue that GCL is essentially a Positive-Unlabeled (PU) learning problem, where the definition of self-supervised tasks should be semantically guided, i.e., augmented samples with similar semantics are considered positive, while others, with unknown semantics, are treated as unlabeled. From this perspective, the key lies in how to extract semantic information. To achieve this, we propose IFL-GCL, using InfoNCE as a "free lunch" to extract semantic information. Specifically, We first prove that under InfoNCE, the representation similarity of node pairs aligns with the probability that the corresponding contrastive sample is positive. Then we redefine the maximum likelihood objective based on the corrected samples, leading to a new InfoNCE loss function. Extensive experiments on both the graph pretraining framework and LLM as an enhancer show significantly improvements of IFL-GCL in both IID and OOD scenarios, achieving up to a 9.05% improvement, validating the effectiveness of semantically guided. Code for IFL-GCL is publicly available at: https://github.com/Camel-Prince/IFL-GCL.
More Than Just A Conversation: A Multi-agent Reasoning Graph Knowledge Distillation for Conversational Stance Detection
Authors: Bingbing Wang, Zhixin Bai, Qianlong Wang, Jingjie Lin, Min Yang, Xi Zeng, Ruifeng Xu
Venue: SIGIR
Year: 2025

Conversational stance detection, which aims to identify stances within conversation threads, has become a research hotspot recently. As the number of dialogue turns increases and the conversation content becomes more complex, existing methods that simply incorporate conversational context are insufficient to effectively capture the nuanced information necessary for accurate stance detection. To address this issue, we introduce a Multi-agent Reasoning Graph Knowledge Distillation (MRGKD) framework, leveraging conversational reasoning among multiple Large Language Models (LLMs) into smaller language models. Specifically, we first construct a multi-agent reasoning graph to infer implicit logical relationships within the conversational history from the diverse perspectives of multiple LLMs. To fully leverage the in-context learning capabilities of LLMs, we design a reasoning knowledge editing mechanism that internalizes new information by aligning the output distribution of smaller language models with both the conversational history and the knowledge derived from the multi-agent reasoning graph. Additionally, we incorporate a contrastive loss to distinguish between correct and incorrect reasoning, alongside a stance detection loss, to fine-tune the smaller language models. This approach not only ensures the accurate acquisition of logical knowledge but also preserves the integrity of the conversational history. Experiments conducted on two public datasets demonstrate that our MRGKD significantly outperforms all baselines.
ProtChatGPT: Towards Understanding Proteins with Hybrid Representation and Large Language Models
Authors: Chao Wang, Hehe Fan, Ruijie Quan, Lina Yao, Yi Yang
Venue: SIGIR
Year: 2025

Protein research is crucial in various scientific disciplines, but understanding their intricate structure-function relationships remains challenging. Recent advancements in Large Language Models (LLMs) have significantly improved the comprehension of task-specific knowledge, suggesting the potential for specialized ChatGPT-like systems in protein research to aid fundamental investigations. In this work, we introduce ProtChatGPT, which aims to learn and understand protein structures using natural language. ProtChatGPT enables users to upload proteins, ask questions, and engage in interactive conversations to produce comprehensive answers. The system comprises multi-level protein encoding, protein-language alignment, and instruction tuning of LLMs. A protein first undergoes multiple protein encoders and PLP-former to produce multi-level hybrid protein embeddings, which are then aligned through a Protein Context Gating (PCG) module with contrastive learning, and projected by an adapter to conform with the LLM. The LLM finally combines user questions with projected protein embeddings to generate informative answers. Experiments show that ProtChatGPT can produce promising responses to proteins and the corresponding user questions. We hope that ProtChatGPT could form the basis for further exploration and application in protein research. Code and our pre-trained model will be publicly available.
MSL: Not All Tokens Are What You Need for Tuning LLM as a Recommender
Authors: Bohao Wang, Feng Liu, Jiawei Chen, Xingyu Lou, Changwang Zhang, Jun Wang, Yuegang Sun, Yan Feng, Chun Chen, Can Wang
Venue: SIGIR
Year: 2025

Large language models (LLMs), known for their comprehension capabilities and extensive knowledge, have been increasingly applied to recommendation systems (RS). Given the fundamental gap between the mechanism of LLMs and the requirement of RS, researchers have focused on fine-tuning LLMs with recommendation-specific data to enhance their performance. Language Modeling Loss (LML), originally designed for language generation tasks, is commonly adopted. However, we identify two critical limitations of LML: 1) it exhibits significant divergence from the recommendation objective; 2) it erroneously treats all fictitious item descriptions as negative samples, introducing misleading training signals. To address these limitations, we propose a novel Masked Softmax Loss (MSL) tailored for fine-tuning LLMs on recommendation. MSL improves LML by identifying and masking invalid tokens that could lead to fictitious item descriptions during loss computation. This strategy can effectively avoid the interference from erroneous negative signals and ensure well alignment with the recommendation objective supported by theoretical guarantees. During implementation, we identify a potential challenge related to gradient vanishing of MSL. To overcome this, we further introduce the temperature coefficient and propose an Adaptive Temperature Strategy (ATS) that adaptively adjusts the temperature without requiring extensive hyperparameter tuning. Extensive experiments conducted on four public datasets further validate the effectiveness of MSL, achieving an average improvement of 42.24% in NDCG@10. The code is available at https://github.com/WANGBohaO-jpg/MSL.
FIM: Frequency-Aware Multi-View Interest Modeling for Local-Life Service Recommendation
Authors: Guoquan Wang, Qiang Luo, Weisong Hu, Pengfei Yao, Wencong Zeng, Guorui Zhou, Kun Gai
Venue: SIGIR
Year: 2025

People's daily lives involve numerous periodic behaviors, such as eating and traveling. Local-life platforms cater to these recurring needs by providing essential services tied to daily routines. Therefore, users' periodic intentions are reflected in their interactions with the platforms. There are two main challenges in modeling users' periodic behaviors in the local-life service recommendation systems: 1) the diverse demands of users exhibit varying periodicities, which are difficult to distinguish as they are mixed in the behavior sequences; 2) the periodic behaviors of users are subject to dynamic changes due to factors such as holidays and promotional events. Existing methods struggle to distinguish the periodicities of diverse demands and overlook the importance of dynamically capturing changes in users' periodic behaviors. To this end, we employ a Frequency-Aware Multi-View Interest Modeling framework (FIM). Specifically, we propose a multi-view search strategy that decomposes users' demands from different perspectives to separate their various periodic intentions. This allows the model to comprehensively extract their periodic features than category-searched-only methods. Moreover, we propose a frequency-domain perception and evolution module. This module uses the Fourier Transform to convert users' temporal behaviors into the frequency domain, enabling the model to dynamically perceive their periodic features. Extensive offline experiments demonstrate that FIM achieves significant improvements on public and industrial datasets, showing its capability to effectively model users' periodic intentions. Furthermore, the model has been deployed on the Kuaishou local-life service platform. Through online A/B experiments, the transaction volume has been significantly improved.
FROG: Effective Friend Recommendation in Online Games via Modality-aware User Preferences
Authors: Qiwei Wang, Dandan Lin, Wenqing Lin, Ziming Wu
Venue: SIGIR
Year: 2025

Due to the convenience of mobile devices, the online games have become an important part for user entertainments in reality, creating a demand for friend recommendation in online games. However, none of existing approaches can effectively incorporate the multi-modal user features (e.g., images and texts) with the structural information in the friendship graph, due to the following limitations: (1) some of them ignore the high-order structural proximity between users (2) some fail to learn the pairwise relevance between users at modality-specific level, and (3) some cannot capture both the local and global user preferences on different modalities. By addressing these issues, in this paper, we propose an end-to-end model FROG that better models the user preferences on potential friends. Comprehensive experiments on both offline evaluation and online deployment at Tencent have demonstrated the superiority of FROG over existing approaches. The source code of this paper can be found at https://github.com/socialalgo/FROG.