OpenRaiser/Intern-Atlas huggingface dataset:
Scaling the Vocabulary of Non-autoregressive Models for Fast Generative Retrieval
Authors: Ravisri Valluri, Akash Kumar Mohankumar, Kushal Dave, Amit Singh, Jian Jiao, Manik Varma, Gaurav Sinha
Venue: KDD
Year: 2025

Generative Retrieval introduces a new approach to Information Retrieval by reframing it as a constrained generation task, leveraging recent advancements in Autoregressive (AR) language models. However, AR-based Generative Retrieval methods suffer from high inference latency and cost compared to traditional dense retrieval techniques, limiting their practical applicability. This paper investigates fully Non-autoregressive (NAR) language models as a more efficient alternative for generative retrieval. While standard NAR models alleviate latency and cost concerns, they exhibit a significant drop in retrieval performance (compared to AR models) due to their inability to capture dependencies between target tokens. To address this, we question the conventional choice of limiting the target token space to solely words or sub-words. We propose PIXNAR, a novel approach that expands the target vocabulary of NAR models to include multi-word entities and common phrases (up to 5 million tokens), thereby reducing token dependencies. PIXNAR employs inference optimization strategies to maintain low inference latency despite the significantly larger vocabulary. Our results demonstrate that PIXNAR achieves a relative improvement of 31.0% in MRR@10 on MS MARCO and 23.2% in Hits@5 on Natural Questions compared to standard NAR models with similar latency and cost. Furthermore, online A/B experiments on a large commercial search engine show significant increase in clicks and revenue.
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model Powered Software (FMware)
Authors: Kirill Vasilevski, Gopi Krishnan Rajbahadur, Gustavo Ansaldi Oliva, Benjamin Rombaut, Keheliya Gallaba, Filipe Roseiro Côgo, Jiahuei (Justina) Lin, Dayi Lin, Haoxiang Zhang, Bouyan Chen, Kishanthan Thangarajah, Ahmed E. Hassan, Zhen Ming (Jack) Jiang
Venue: KDD
Year: 2025

Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components. In this KDD 2025 tutorial, we present a comprehensive exploration of FMware that combines a curated catalogue of challenges with real-world production concerns. We first discuss the state of research and practice in building FMware. We further examine the difficulties in selecting suitable models, aligning high-quality domain-specific data, engineering robust prompts, and orchestrating autonomous agents. We then address the complex journey from impressive demos to production-ready systems by outlining issues in system testing, optimization, deployment, and integration with legacy software. Drawing on our industrial experience and recent research in the area, we provide actionable insights and a technology roadmap for overcoming these challenges. Attendees will gain practical strategies to enable the creation of trustworthy FMware in the evolving technology landscape.
Bayesian Stream Tuner: Dynamic Hyperparameter Optimization for Real-Time Data Streams
Authors: Nilesh Verma, Albert Bifet, Bernhard Pfahringer, Maroua Bahri
Venue: KDD
Year: 2025

Hyperparameter optimization is crucial for maximizing machine learning model performance, yet most existing algorithms are designed for batch or offline scenarios and assume static data distributions. Such assumptions fall short in data stream settings, where models must adapt to evolving inputs in real time. To address these limitations, we propose the Bayesian Stream Tuner (BST), a novel framework for online hyperparameter optimization in non-stationary data streams. BST maintains a dynamic set of candidate hyperparameter configurations and periodically refines them using an incremental Bayesian model, which estimates configuration performance based on recent data statistics and hyperparameter values. This systematic exploration and refinement strategy allows BST to detect and respond to concept drift by resetting its adaptation mechanisms whenever necessary, ensuring strong performance under changing distributions. Our theoretical analysis establishes sublinear regret bounds for BST in dynamic environments, and extensive experiments on classification and regression tasks demonstrate that BST consistently outperforms state-of-the-art online hyperparameter optimization methods in both predictive accuracy and adaptability, making it a powerful solution for real-time hyperparameter tuning in evolving data streams.
An Audio-centric Multi-task Learning Framework for Streaming Ads Targeting on Spotify
Authors: Shivam Verma, Vivian Chen, Darren Mei
Venue: KDD
Year: 2025

Spotify, a large-scale multimedia platform, attracts over 675 million monthly active users who collectively consume millions of hours of music, podcasts, audiobooks, and video content. This diverse content consumption pattern introduces unique challenges for computational advertising, which must effectively integrate a variety of ad modalities including audio, video, and display-within a single user experience. Traditional ad recommendation models, primarily designed for foregrounded experiences, often struggle to reconcile the platform's inherent audio-centrality with the demands of optimizing ad performance across multiple formats and modalities. To overcome these challenges, we introduce Cross-modal Adaptive Mixture-of-Experts (CAMoE), a novel framework for optimizing click-through rate (CTR) prediction in both audio-centric and multi-modal settings. CAMoE enhances traditional mixture-of-experts (MoE) models by incorporating modality-aware task grouping, adaptive loss masking, and deep-cross networks (DCN) to capture complex feature interactions within a multi-modal ad ecosystem. Through extensive ablation studies, we demonstrate that this approach achieves near Pareto-optimal performance across audio, video, and display ad formats, significantly improving AUC-PR compared to conventional single-task and content-based multi-task learning baselines. When deployed at scale on Spotify's ad serving platform, CAMoE delivered substantial gains, yielding a 14.5% increase in CTR for audio ads, a 1.3% increase for video ads, and a 4.8% reduction in expected cost-per-click (eCPC) for audio slots.
Matryoshka Model Learning for Improved Elastic Student Models
Authors: Chetan Verma, Aditya Srinivas Timmaraju, Cho-Jui Hsieh, Suyash Damle, Ngot Bui, Yang Zhang, Wen Chen, Xin Liu, Prateek Jain, Inderjit S. Dhillon
Venue: KDD
Year: 2025

Industry-grade ML models are carefully designed to meet rapidly evolving serving constraints, which requires significant resources for model development. In this paper, we propose MatTA, a framework for training multiple accurate Student models using a novel Teacher-TA-Student recipe. TA models are larger versions of the Student models with higher capacity, and thus allow Student models to better relate to the Teacher model and also bring in more domain-specific expertise. Furthermore, multiple accurate Student models can be extracted from the TA model. Therefore, despite only one training run, our methodology provides multiple servable options to trade off accuracy for lower serving cost. We demonstrate the proposed method, MatTA, on proprietary datasets and models. Its practical efficacy is underscored by live A/B tests within a production ML system, demonstrating 20% improvement on a key metric. We also demonstrate our method on GPT-2 Medium, a public model, and achieve relative improvements of over 24% on SAT Math and over 10% on the LAMBADA benchmark.
Dynamic Causal Structure Discovery and Causal Effect Estimation
Authors: Jianian Wang, Rui Song
Venue: KDD
Year: 2025

To represent the causal relationships between variables, a directed acyclic graph (DAG) is widely utilized in many areas, such as social sciences, epidemics, and genetics. Many causal structure learning approaches are developed to learn the hidden causal structure using deep learning approaches. However, these approaches have a hidden assumption that the causal relationship remains unchanged over time, which may not hold in real life. In this paper, we develop a new framework to model the dynamic causal graph where the causal relations are allowed to be time-varying. We incorporate the basis approximation method into the score-based causal discovery approach to capture the dynamic pattern of causal graphs. Utilizing the autoregressive model structure, we could capture both contemporaneous and time-lagged causal relationships while allowing them to vary with time. We propose an algorithm that could provide both past-time estimates and future-time predictions on the causal graphs, and conduct simulations to demonstrate the usefulness of the proposed method. We also apply the proposed method for the covid-data analysis, and provide causal estimates on how the effect of policy restriction changes.
FedSC: Federated Learning with Semantic-Aware Collaboration
Authors: Huan Wang, Haoran Li, Huaming Chen, Jun Yan, Jiahua Shi, Jun Shen
Venue: KDD
Year: 2025

Federated learning (FL) aims to train models collaboratively across clients without sharing data for privacy-preserving. However, one major challenge is the data heterogeneity issue, which refers to the biased labeling preferences at multiple clients. A number of existing FL methods attempt to tackle data heterogeneity locally (e.g., regularizing local models) or globally (e.g., fine-tuning global model), often neglecting inherent semantic information contained in each client. To explore the possibility of using intra-client semantically meaningful knowledge in handling data heterogeneity, in this paper, we propose Federated Learning with Semantic-Aware Collaboration (FedSC) to capture client-specific and class-relevant knowledge across heterogeneous clients. The core idea of FedSC is to construct relational prototypes and consistent prototypes at semantic-level, aiming to provide fruitful class underlying knowledge and stable convergence signals in a prototype-wise collaborative way. On the one hand, FedSC introduces an inter-contrastive learning strategy to bring instance-level embeddings closer to relational prototypes with the same semantics and away from distinct classes. On the other hand, FedSC devises consistent prototypes via a discrepancy aggregation manner, as a regularization penalty to constrain the optimization region of the local model. Moreover, a theoretical analysis for FedSC is provided to ensure a convergence guarantee. Experimental results on various challenging scenarios demonstrate the effectiveness of FedSC and the efficiency of crucial components.
GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction
Authors: Rongzheng Wang, Shuang Liang, Qizhi Chen, Jiasheng Zhang, Ke Qin
Venue: KDD
Year: 2025

Large language models (LLMs) have been demonstrated to possess the capabilities to understand fundamental graph properties and address various graph reasoning tasks. Existing methods fine-tune LLMs to understand and execute graph reasoning tasks by specially designed task instructions. However, these Text-Instruction methods generally exhibit poor performance. Inspired by tool learning, researchers propose Tool-Instruction methods to solve various graph problems by special tool calling (e.g., function, API and model), achieving significant improvements in graph reasoning tasks. Nevertheless, current Tool-Instruction approaches focus on the tool information and ignore the graph structure information, which leads to significantly inferior performance on small-scale LLMs (less than 8B). To tackle this issue, we propose GraphTool-Instruction, an innovative Instruction-tuning approach that decomposes the graph reasoning task into three distinct subtasks (i.e., graph extraction, tool name identification and tool parameter extraction), and design specialized instructions for each subtask. Our GraphTool-Instruction can be used as a plug-and-play prompt for different LLMs without fine-tuning. Moreover, building on GraphTool-Instruction, we develop GTools, a dataset that includes twenty graph reasoning tasks, and create a graph reasoning LLM called GraphForge based on Llama3-8B. We conduct extensive experiments on twenty graph reasoning tasks with different graph types (e.g., graph size or graph direction), and we find that GraphTool-Instruction achieves SOTA compared to Text-Instruction and Tool-Instruction methods. Fine-tuned on GTools, GraphForge gets further improvement of over 30% compared to the Tool-Instruction enhanced GPT-3.5-turbo, and it performs comparably to the high-cost GPT-4o. Our codes and data are available at https://github.com/RongzhengWang/GraphTool-Instruction.
Improved Accuracy, Declining Orders: Uncovering the Vicious Competition Trap in Multi-Domain Recommendation
Authors: Dan Wang, Zijian Song, Lifang Deng, Yihuan Wu, Jin Niu, Jiandong Zhang, Kaigui Bian, Bin Cui
Venue: KDD
Year: 2025

E-commerce recommendation platforms often operate across multiple domains, giving rise to Multi-Domain Recommendation (MDR) methods. These methods leverage auxiliary information from other domains to capture comprehensive user preferences and enhance recommendation accuracy. However, cross-domain knowledge transfer often causes domains to capture similar preferences. Additionally, most MDR models optimize for next-item prediction, which promotes greedy strategies focused on immediate gains within individual domains while neglecting the platform-wide profit. These two factors result in vicious competition, as our empirical analysis reveals: Domains recommend overlapping item lists, competing for the same pool of user attention and platform traffic, reducing the overall orders. To address this, we propose the Direct Preference Optimization from Manual Rules (DPOMR) training paradigm for MDR. Inspired by Reinforcement Learning from Human Feedback (RLHF), DPOMR learns user preferences from pairwise preference data generated through manual rules rooted in business expertise. Furthermore, we design a variant of Direct Preference Optimization (DPO) to bypass expensive RL training. Extensive experiments demonstrate the advantages of DPOMR, including: (1) achieving SOTA performance in boosting overall orders(2) showing compatibility with various backbones, and (3) offering robust extendibility to integrate diverse manual rules tailored to specific business needs.
Generative Next POI Recommendation with Semantic ID
Authors: Dongsheng Wang, Yuxi Huang, Shen Gao, Yifan Wang, Chengrui Huang, Shuo Shang
Venue: KDD
Year: 2025

Point-of-interest (POI) recommendation systems aim to predict the next destinations of user based on their preferences and historical check-ins. Existing generative POI recommendation methods usually employ random numeric IDs for POIs, limiting the ability to model semantic relationships between similar locations. In this paper, we propose Generative Next POI Recommendation with Semantic ID (GNPR-SID), an LLM-based POI recommendation model with a novel semantic POI ID (SID) representation method that enhances the semantic understanding of POI modeling. There are two key components in our GNPR-SID: (1) a Semantic ID Construction module that generates semantically rich POI IDs based on semantic and collaborative features, and (2) a Generative POI Recommendation module that fine-tunes LLMs to predict the next POI using these semantic IDs. By incorporating user interaction patterns and POI semantic features into the semantic ID generation, our method improves the recommendation accuracy and generalization of the model. To construct semantically related SIDs, we propose a POI quantization method based on residual quantized variational autoencoder, which maps POIs into a discrete semantic space. We also propose a diversity loss to ensure that SIDs are uniformly distributed across the semantic space. Extensive experiments on three benchmark datasets demonstrate that GNPR-SID substantially outperforms state-of-the-art methods, achieving up to 16% improvement in recommendation accuracy.
A Framework for Evaluating AI Agents in Open-Ended Conversations via Scripted Simulation
Authors: Clarice Wang, Yimin Shi, Xiaokui Xiao
Venue: KDD
Year: 2025

Traditional evaluations of conversational AI have primarily focused on ''closed-ended'' interactions, where a human user queries the AI system, such as in customer support. However, many advanced real-world applications-such as job interviews, podcast hosting, and legal or healthcare intake discussions-require ''open-ended'' interactions in which the AI must take initiative by formulating questions to fully understand the human user's story. As AI assumes broader roles that demand greater autonomy, evaluating its performance in open-ended conversations becomes significantly more complex. This paper introduces a novel framework for rigorously assessing AI agents in such open-ended interviews. In this framework, a secondary AI agent (Agent B) simulates a human interviewee by strictly following a structured script that defines topic strengths and weaknesses, along with guidelines for when to hint at or deviate from these topics. Meanwhile, the primary AI agent (Agent A) engages in dynamic questioning to uncover the script's underlying facts and narrative. By comparing the final conversation transcript to the original script, we assess Agent A using metrics such as completeness, consistency, and investigative depth. This approach not only establishes a new benchmark for open-ended conversational skills but also provides insight into how effectively an AI agent can detect and navigate strategic diversions in scripted behavior.
Noise-Resilient Point-wise Anomaly Detection in Time Series Using Weak Segment Labels
Authors: Yaxuan Wang, Hao Cheng, Jing Xiong, Qingsong Wen, Han Jia, Ruixuan Song, Liyuan Zhang, Zhaowei Zhu, Yang Liu
Venue: KDD
Year: 2025

Detecting anomalies in temporal data has gained significant attention across various real-world applications, aiming to identify unusual events and mitigate potential hazards. In practice, situations often involve a mix of segment-level labels (detected abnormal events with segments of time points) and unlabeled data (undetected events), while the ideal algorithmic outcome should be point-level predictions. Therefore, the huge label information gap between training data and targets makes the task challenging. In this study, we formulate the above imperfect information as noisy labels and propose NRdetector, a noise-resilient framework that incorporates confidence-based sample selection, robust segment-level learning, and data-centric point-level detection for multivariate time series anomaly detection. Particularly, to bridge the information gap between noisy segment-level labels and missing point-level labels, we develop a novel loss function that can effectively mitigate the label noise and consider the temporal features. It encourages the smoothness of consecutive points and the separability of points from segments with different labels. Extensive experiments on real-world multivariate time series datasets with 11 different evaluation metrics demonstrate that NRdetector consistently achieves robust results across multiple real-world datasets, outperforming various baselines adapted to operate in our setting.
PDMC: Generating Feasible Algorithmic Recourse via Perturbation Data Manifold Constraint
Authors: Zimu Wang, Hao Zou, Han Yu, Shaohua Fan, Haotian Wang, Yue He, Peng Cui
Venue: KDD
Year: 2025

To provide actionable insights and interpretations for individuals affected by algorithmic decisions, algorithmic recourse-demonstrating how outcomes change with modifications to input features-is introduced to facilitate outcome adjustment. However, existing studies often focus on different notions of feasibility and impose complex optimization constraints, relying on strong assumptions and expert knowledge that may be impractical or not widely applicable. In this paper, we propose leveraging adherence to the perturbation data manifold to model typical feasibility challenges, providing both a theoretical clarification and a practical framework. We design optimization constraints based on this model and introduce our method, the Perturbation Data Manifold Constraint (PDMC), to ensure the feasibility of generated algorithmic recourses. Through extensive experiments on both simulated and real clinical data, we validate the rationale and effectiveness of PDMC.
Non-exchangeable Conformal Prediction for Temporal Graph Neural Networks
Authors: Tuo Wang, Jian Kang, Yujun Yan, Adithya Kulkarni, Dawei Zhou
Venue: KDD
Year: 2025

Conformal prediction for graph neural networks (GNNs) offers a promising framework for quantifying uncertainty, enhancing GNN reliability in high-stakes applications. However, existing methods predominantly focus on static graphs, neglecting the evolving nature of real-world graphs. Temporal dependencies in graph structure, node attributes, and ground truth labels violate the fundamental exchangeability assumption of standard conformal prediction methods, limiting their applicability. To address these challenges, in this paper, we introduce NCPNET, a novel end-to-end conformal prediction framework tailored for temporal graphs. Our approach extends conformal prediction to dynamic settings, mitigating statistical coverage violations induced by temporal dependencies. To achieve this, we propose a diffusion-based non-conformity score that captures both topological and temporal uncertainties within evolving networks. Additionally, we develop an efficiency-aware optimization algorithm that improves the conformal prediction process, enhancing computational efficiency and reducing coverage violations. Extensive experiments on diverse real-world temporal graphs, including WIKI, REDDIT, DBLP, and IBM Anti-Money Laundering dataset, demonstrate NCPNET's capability to ensure guaranteed coverage in temporal graphs, achieving up to a 31% reduction in prediction set size on the WIKI dataset, significantly improving efficiency compared to state-of-the-art methods. Our data and code are available at https://github.com/ODYSSEYWT/NCPNET.
Producer-Side Experiments Based on Counterfactual Interleaving Designs for Online Recommender Systems
Authors: Yan Wang, Shan Ba
Venue: KDD
Year: 2025

Recommender systems play a crucial role in online platforms, providing personalized recommendations for purchases, content consumption, and interpersonal connections. These systems involve two sides: producers (sellers, content creators, service providers, etc.) and consumers (buyers, viewers, customers, etc.). To optimize online recommender systems, A/B tests serve as the golden standard for comparing different ranking models and evaluating their impacts on both sides. While consumer-side experiments are relatively straightforward to design and commonly employed to assess ranking changes' effects on the behavior of consumers (buyers, viewers, etc.), designing producer-side experiments for an online recommender/ranking system is notably more complex. This complexity arises from the necessity of ranking producer items in the treatment and control groups by different models and then merging them into a unified ranking for presentation to each consumer. Existing design solutions in the literature lack rigorous guiding principles, leading to ad hoc approaches. In this paper, we address the limitations of current methods and propose the principles of consistency and monotonicity for designing producer-side experiments in online recommender systems. Building upon these principles, we also present a systematic solution based on counterfactual interleaving designs to accurately measure the impacts of ranking changes on the producers (sellers, content creators, etc.).