OpenRaiser/Intern-Atlas huggingface dataset:
Beyond Item Dissimilarities: Diversifying by Intent in Recommender Systems
Authors: Yuyan Wang, Cheenar Banerjee, Samer Chucri, Fabio Soldo, Sriraj Badam, Ed H. Chi, Minmin Chen
Venue: KDD
Year: 2025

It has become increasingly clear that recommender systems that overly focus on short-term engagement prevents users from exploring diverse interests, ultimately hurting long-term user experience. To tackle this challenge, numerous diversification algorithms have been proposed as the final stage of recommender systems. These algorithms typically rely on measures of item similarity, aiming to maximize the dissimilarity across items in the final set of recommendations. However, in this work, we demonstrate the benefits of going beyond item-level similarities by utilizing higher-level user understanding-specifically, user intents that persist across multiple interactions or recommendation sessions-in diversification. Our approach is motivated by the observation that user behaviors on online platforms are largely driven by their underlying intents. Therefore, final recommendations should ensure that a diverse set of user intents is accurately represented. While user intent has primarily been studied in the context of search, it is less clear how to incorporate real-time dynamic intent predictions in recommender systems. To address this gap, we develop a probabilistic intent-based whole-page diversification framework for the final stage of a recommender system. Starting with a prior belief of user intents, the proposed framework sequentially selects items for each position based on these beliefs and subsequently updates posterior beliefs about the intents. This approach ensures that different user intents are represented on a page, towards optimizing long-term user experience. We experiment with the intent diversification framework on YouTube, the world's largest video recommendation platform, serving billions of users daily. Live experiments on a diverse set of intents show that the proposed framework increases Daily Active Users (DAU) and overall user enjoyment, validating its effectiveness in facilitating long-term planning. Specifically, it enables users to consistently discover and engage with diverse content that aligns with their underlying intents over time, leading to an improved long-term user experience.
HoME: Hierarchy of Multi-Gate Experts for Multi-Task Learning at Kuaishou
Authors: Xu Wang, Jiangxia Cao, Zhiyi Fu, Kun Gai, Guorui Zhou
Venue: KDD
Year: 2025

In this paper, we present the practical problems and the lessons learned at short-video services from Kuaishou. In industry, a widely-used multi-task framework is the Mixture-of-Experts (MoE) paradigm, which always introduces some shared and specific experts for each task and then uses gate networks to measure related experts' contributions. Although the MoE achieves remarkable improvements, we still observe three anomalies that seriously affect model performances in our iteration: (1) Expert Collapse: We found that experts' output distributions are significantly different, and some experts have over 90% zero activations with ReLU, making it hard for gate networks to assign fair weights to balance experts. (2) Expert Degradation: Ideally, the shared-expert aims to provide predictive information for all tasks simultaneously. Nevertheless, we find that some shared-experts are occupied by only one task, which indicates that shared-experts lost their ability but degenerated into some specific-experts. (3) Expert Underfitting: In our services, we have dozens of behavior tasks that need to be predicted, but we find that some data-sparse prediction tasks tend to ignore their specific-experts and assign large weights to shared-experts. The reason might be that the shared-experts can perceive more gradient updates and knowledge from dense tasks, while specific-experts easily fall into underfitting due to their sparse behaviors. Motivated by those observations, we propose HoME to achieve a simple, efficient and balanced MoE system for multi-task learning. Specifically, we conduct three insightful modifications: (1) Expert normalization&Swish mechanism to align expert output distributions and avoid expert collapse. (2) Hierarchy mask mechanism to enhance sharing efficiency between tasks to reduce occupancy issues and away from expert degradation. (3) Feature-gate&Self-gate mechanisms to ensure each expert could obtain appropriate gradient to maximize its effectiveness. To our knowledge, this paper is the first work to focus on improving multi-task MoE system stability, and we conduct extensive offline&online (average improves 0.52% GAUC offline & 0.954% play-time per user online) experiments and ablation analyses to demonstrate our HoME effectiveness. From March 2024, our HoME has been widely deployed on various services at Kuaishou, supporting 400 Million active users daily.
FoodGPT: Reinforcement Post-Training of Large Language Models in the Food Delivery Domain
Authors: Jiang Wang, Zhengxin Dong, Bing Bai, Guyu Jiang, Aiquan Yuan, Guodong Cao
Venue: KDD
Year: 2025

On-demand Food Delivery (OFD) platforms, such as Ele.me and Meituan, have transformed daily life by offering convenient ordering services. However, challenges remain in understanding user intentions and processing product-related text information. Existing NLP models, while advanced in general tasks, are less effective for OFD-specific needs due to data scarcity and high computational costs. This paper introduces FoodInstruct, a Chinese dataset with 1.6 million examples across 12 OFD-related NLP tasks, and FoodGPT, a domain-specific large language model. We propose an efficient reinforcement post-training framework that combines Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Proximal Policy Optimization (PPO) with an additional rule-based reward signal. The resulting foundation model, FoodGPT, enhances model performance while minimizing resource consumption. Experimental results demonstrate that FoodGPT outperforms general models, such as Qwen 2.5, Llama3.1 and DeepSeek-LLM, on OFD tasks, using fewer data and training iterations. The model has already been deployed across numerous applications within the company. For instance, it achieved a 0.57% increase in click-through rate and a 0.32% increase in user visit-to-purchase rate in the ITG online experiment. The core contributions are now publicly accessible at https://huggingface.co/elemenlp.
TayFCS: Towards Light Feature Combination Selection for Deep Recommender Systems
Authors: Xianquan Wang, Zhaocheng Du, Jieming Zhu, Chuhan Wu, Qinglin Jia, Zhenhua Dong
Venue: KDD
Year: 2025

Feature interaction modeling is crucial for deep recommendation models. A common and effective approach is to construct explicit feature combinations to enhance model performance. However, in practice, only a small fraction of these combinations are truly informative. Thus it is essential to select useful feature combinations to reduce noise and manage memory consumption. While feature selection methods have been extensively studied, they are typically limited to selecting individual features. Extending these methods for high-order feature combination selection presents a significant challenge due to the exponential growth in time complexity when evaluating feature combinations one by one. In this paper, we propose TayFCS, a lightweight feature combination selection method that significantly improves model performance. Specifically, we propose the Taylor Expansion Scorer (TayScorer) module for field-wise Taylor expansion on the base model. Instead of evaluating all potential feature combinations' importance by repeatedly running experiments with feature adding and removal, this scorer only needs to approximate them based on their sub-components' gradients. They can be simply computed with one backward pass based on a trained recommendation model. To further reduce information redundancy between feature combinations and their sub-components, we introduce Logistic Regression Elimination (LRE) that estimates the information gain of feature combinations over their sub-components based on the above importance scores. Experimental results on three benchmark datasets validate both the effectiveness and efficiency of our approach. Furthermore, online A/B test results demonstrate its practical applicability and commercial value.
Exploiting Student Parallelism for Low-latency GPU Inference of BERT-like Models in Online Services
Authors: Weiyan Wang, Yilun Jin, Yiming Zhang, Victor Junqiu Wei, Han Tian, Li Chen, Jinbao Xue, Yangyu Tao, Di Wang, Kai Chen
Venue: KDD
Year: 2025

BERT-like models have been widely adopted in text mining and web search due to their high accuracy. However, large BERT-like models suffer from inefficient online inference on GPUs for two main reasons. First, their high accuracy relies on large model depth, which linearly increases sequential computation on GPUs. Second, stochastic and dynamic online workloads lead to extra costs due to batching and padding. To address the problem, we present Student Parallelism for efficient GPU inference of BERT-like models under real-world online workloads. At its core, Student Parallelism adopts stacking distillation and boosting ensemble, distilling the original deep model into a group of shallow but virtually stacked student models running in parallel. This enables Student Parallelism to achieve a low model depth (e.g., two layers), and thus low inference latency while maintaining accuracy. In addition, we design adaptive student pruning to adjust the number of students according to the dynamic online workloads. For example, during workload bursts, it can temporarily decrease the number of students with minimal accuracy loss to improve system throughput. Extensive experiments on real-world datasets and workloads show that Student Parallelism achieves up to 4.1× lower latency while maintaining accuracy and up to 22.27× higher throughput during workload bursts.
Personalized Query Auto-Completion for Long and Short-Term Interests with Adaptive Detoxification Generation
Authors: Zhibo Wang, Xiaoze Jiang, Zhiheng Qin, Enyun Yu
Venue: KDD
Year: 2025

Query auto-completion (QAC) plays a crucial role in modern search systems. However, in real-world applications, there are two pressing challenges that still need to be addressed. First, there is a need for hierarchical personalized representations for users. Previous approaches have typically used users' search behavior as a single, overall representation, which proves inadequate in more nuanced generative scenarios. Additionally, query prefixes are typically short and may contain typos or sensitive information, increasing the likelihood of generating toxic content compared to traditional text generation tasks. Such toxic content can degrade user experience and lead to public relations issues. Therefore, the second critical challenge is detoxifying QAC systems. Recent efforts to mitigate toxicity have involved generating queries unrelated to the given prefix, leading this approach still negatively impacts user experience. To address these two limitations, we propose a novel model (LaD) that captures personalized information from both long-term and short-term interests, incorporating adaptive detoxification. In LaD, personalized information is captured hierarchically at both coarse-grained and fine-grained levels. This approach preserves as much personalized information as possible while enabling online generation within time constraints. To move a further step, we propose an online training method based on Reject Preference Optimization (RPO). By incorporating a special token [Reject] during both the training and inference processes, the model achieves adaptive detoxification. Consequently, the generated text presented to users is both non-toxic and relevant to the given prefix. We conduct comprehensive experiments on industrial-scale datasets and perform online A/B tests, delivering the largest single-experiment metric improvement in nearly two years of our product. Our model has been deployed on Kuaishou search, driving the primary traffic for hundreds of millions of active users. The code is available at https://github.com/JXZe/LaD.
A Survey on Small Language Models in the Era of Large Language Models: Architecture, Capabilities, and Trustworthiness
Authors: Fali Wang, Minhua Lin, Yao Ma, Hui Liu, Qi He, Xianfeng Tang, Jiliang Tang, Jian Pei, Suhang Wang
Venue: KDD
Year: 2025

Large language models (LLMs) based on Transformer architecture are powerful but face challenges with deployment, inference latency, and costly fine-tuning. These limitations highlight the emerging potential of small language models (SLMs), which can either replace LLMs through innovative architectures and technologies, or assist them as efficient proxy or reward models. Emerging architectures such as Mamba and xLSTM address the quadratic scaling of inference with window length in Transformers by enabling linear scaling. To maximize SLM performance, test-time compute scaling strategies reduce the performance gap with LLMs by allocating extra compute budget during test time. Beyond standalone usage, SLMs could also assist in LLMs via weak-to-strong learning, proxy tuning, and guarding, fostering secure and efficient LLM deployment. Lastly, the trustworthiness of SLMs remains a critical yet underexplored research area. However, there is a lack of tutorials on cutting-edge SLM technologies, prompting us to conduct one.
Pre-training Time Series Models with Stock Data Customization
Authors: Mengyu Wang, Tiejun Ma, Shay B. Cohen
Venue: KDD
Year: 2025

Stock selection, which aims to predict stock prices and identify the most profitable ones, is a crucial task in finance. While existing methods primarily focus on developing model structures and building graphs for improved selection, pre-training strategies remain underexplored in this domain. Current stock series pre-training follows methods from other areas without adapting to the unique characteristics of financial data, particularly overlooking stock-specific contextual information and the non-stationary nature of stock prices. Consequently, the latent statistical features inherent in stock data are underutilized. In this paper, we propose three novel pre-training tasks tailored to stock data characteristics: stock code classification, stock sector classification, and moving average prediction. We develop the Stock Specialized Pre-trained Transformer (SSPT) based on a two-layer transformer architecture. Extensive experimental results validate the effectiveness of our pre-training methods and provide detailed guidance on their application. Evaluations on five stock datasets, including four markets and two time periods, demonstrate that SSPT consistently outperforms the market and existing methods in terms of both cumulative investment return ratio and Sharpe ratio. Additionally, our experiments on simulated data investigate the underlying mechanisms of our methods, providing insights into understanding price series. Our code is publicly available at: https://github.com/astudentuser/Pre-training-Time-Series-Models-with-Stock-Data-Customization.
Enhanced Emotion-aware Music Recommendation via Large Language Models
Authors: Shijia Wang, Tianpei Ouyang, Yunfan Zhou, Qiang Xiao, Yintao Ren, Yifei Pan, Fangjian Li, Chuanjiang Luo
Venue: KDD
Year: 2025

The profound psychological interplay between music and human affect underscores the critical role of emotion-aware modeling in music recommendation systems. While existing approaches predominantly employ emotion classification or vector representation methods, their capacity to capture nuanced emotional semantics remains limited. This paper presents the Large-Language-Model-based emotion-aware music recommendation model(LEMON), a novel framework that harnesses Large Language Models to address two fundamental challenges: extracting multi-faceted emotional semantics from music content and dynamically aligning users' transient emotional states with long-term preferences. Our framework introduces three key innovations: First, a hierarchical LLM-driven emotion extraction pipeline that decodes complex emotional narratives from lyrics through multi-stage prompt engineering, generating semantically rich song embeddings. Second, a dual-temporal encoder that separately models users' short-term emotion representation derived from recent engagement patterns and long-term preferences extracted from historical interactions. Third, an adaptive fusion coefficient that dynamically balances temporal features using user profile attributes to enable personalized recommendation adaptation. Extensive evaluations on industrial and public datasets demonstrate LEMON's superiority, achieving 4.22% AUC improvements over state-of-the-art baselines. Online A/B tests on NetEase Cloud Music, one of China's largest music streaming platforms validate a 1% increase in user engagement metrics. Ablation studies confirm the necessity of each component, particularly highlighting LLMs' performance gain in capturing context-dependent emotions compared to conventional methods. This work advances the integration of LLMs' semantic reasoning capabilities into emotion-aware recommendation systems, offering practical insights for online music personalization.
PSSketch: Finding Persistent and Sparse Flow with High Accuracy and Efficiency
Authors: Jiayao Wang, Qilong Shi, Xiyan Liang, Han Wang, Wenjun Li, Ziling Wei, Weizhe Zhang, Shuhui Chen
Venue: KDD
Year: 2025

Finding persistent sparse (PS) flow is critical to early warning of various threats. Previous works have predominantly focused on either heavy or persistent flows, with limited attention given to PS flows. Although some recent studies pay attention to PS flows, they struggle to establish an objective criterion due to insufficient data-driven observations, resulting in reduced accuracy. In this paper, we define a new criterion ''anomaly boundary'' to distinguish PS flows from regular flows. Specifically, a flow whose persistence exceeds a threshold will be protected, while a protected flow with a density lower than a threshold is reported as a PS flow. We then introduce PSSketch, a high-precision layered sketch, to find PS flows. PSSketch employs variable-length bitwise counters, where the first layer tracks the frequency and persistence of all flows, and the second layer protects potential PS flows and records overflow counts from the first layer. Some optimizations have also been implemented to reduce memory consumption further and improve accuracy. The experiments show that PSSketch reduces memory consumption by 1-2 orders of magnitude compared to the strawman solution combined with existing work. Compared with SOTA solutions for finding PS flows, it outperforms up to 2.94x higher in F1 score and reduces ARE by 1-2 orders of magnitude. Meanwhile, PSSketch achieves a higher throughput than these solutions.
Mitigating Redundancy in Deep Recommender Systems: A Field Importance Distribution Perspective
Authors: Xianquan Wang, Likang Wu, Zhi Li, Haitao Yuan, Shuanghong Shen, Huibo Xu, Yu Su, Chenyi Lei
Venue: KDD
Year: 2025

In the realm of recommender systems, accurately predicting Click-Through Rate (CTR) is a critical task that involves learning user-item interaction features. Many researchers propose novel models to mine interaction signals, but they neglect that redundancy itself causes high computational cost and leads to suboptimal performance. Some tried to remove redundancy by dropping useless features, or shrinking the size of embedding table. However, current feature selection methods are vulnerable to training stochasticity and data dynamics, while embedding size assignment techniques neglect the importance relationships between feature fields. The simple combination of the two optimization ways will also yield poor performance due to the inherent gap in their optimization targets. Hence, there is no effective paradigm that can optimize feature fields from the two aspects in a simultaneous and coordinated way. In this paper, we identify the core issue as the lack of a practical score to measure the contribution of feature fields, and propose a distribution-based field optimization framework that adopts importance distribution to provide a comprehensive view for both methods. We innovatively design a learner for each field to acquire the stable and comprehensive importance situation. Then, based on this, we eliminate noise features, and assign adaptive embedding sizes for different feature fields according to the similarity of importance. With this field optimization, our proposed framework has extremely low pre-training overhead, greatly reduces training and inference time, and even achieves more accurate prediction results with fewer feature fields.
Scaling Transformers for Discriminative Recommendation via Generative Pretraining
Authors: Chunqi Wang, Bingchao Wu, Zheng Chen, Lei Shen, Bing Wang, Xiaoyi Zeng
Venue: KDD
Year: 2025

Discriminative recommendation tasks, such as CTR (click-through rate) and CVR (conversion rate) prediction, play critical roles in the ranking stage of large-scale industrial recommender systems. However, training a discriminative model encounters a significant overfitting issue induced by data sparsity. Moreover, this overfitting issue worsens with larger models, causing them to underperform smaller ones. To address the overfitting issue and enhance model scalability, we propose a framework named GPSD (Generative Pretraining for Scalable Discriminative Recommendation), drawing inspiration from generative training, which exhibits no evident signs of overfitting. GPSD leverages the parameters learned from a pretrained generative model to initialize a discriminative model, and subsequently applies a sparse parameter freezing strategy. Extensive experiments conducted on both industrial-scale and publicly available datasets demonstrate the superior performance of GPSD. Moreover, it delivers remarkable improvements in online A/B tests. GPSD offers two primary advantages: 1) it substantially narrows the generalization gap in model training, resulting in better test performance; and 2) it leverages the scalability of Transformers, delivering consistent performance gains as models are scaled up. Specifically, we observe consistent performance improvements as the model dense parameters scale from 13K to 0.3B, closely adhering to power laws. These findings pave the way for unifying the architectures of recommendation models and language models, enabling the direct application of techniques well-established in large language models to recommendation models.