OpenRaiser/Intern-Atlas huggingface dataset:
Fairness without Demographics through Learning Graph of Gradients
Authors: Yingtao Luo, Zhixun Li, Qiang Liu, Jun Zhu
Venue: KDD
Year: 2025

Machine learning systems are notoriously prone to biased predictions about certain demographic groups, leading to algorithmic fairness issues. Due to privacy concerns and data quality problems, some demographic information may not be available in the training data and the complex interaction of different demographics can lead to a lot of unknown minority subpopulations, which all limit the applicability of group fairness. Many existing works on fairness without demographics assume the correlation between groups and features. However, we argue that the model gradients are also valuable for fairness without demographics. In this paper, we show that the correlation between gradients and groups can help identify and improve group fairness. With an adversarial weighting architecture, we construct a graph where samples with similar gradients are connected and learn the weights of different samples from it. Unlike the surrogate grouping methods that cluster groups from features and labels as proxy sensitive attribute, our method leverages the graph structure as a soft grouping mechanism, which is much more robust to noises. The results show that our method is robust to noise and can improve fairness significantly without decreasing the overall accuracy too much.
AGODE: Adaptive Graph ODE for Grid-free Fluid Modeling and Domain Adaptation
Authors: Jie Lv, Shuyuan Yang, Zhixi Feng
Venue: KDD
Year: 2025

This paper studies grid-free point process modeling under varying fluid parameters. Existing methods rely on grid-based approaches or fixed parameters, making it challenging to handle complex nonlinear dynamics and out-of-distribution (OOD) scenarios. To address this, we propose Adaptive Perturbation Graph ODE (AGODE), a novel framework that integrates three key innovations: (1) an adaptive conditioning mechanism for physical parameter adaptation(2) a continuous graph neural ODE for spatiotemporal evolution modeling, and (3) a perturbation module with mutual information maximization for uncertainty quantification. AGODE employs graph neural networks to encode unstructured point cloud data into latent dynamics governed by neural ODEs, where physical parameters are injected through context-aware conditioning vectors. The perturbation module generates diverse trajectory samples by introducing stochastic noise during ODE integration, while contrastive learning aligns predictions with physical contexts to filter implausible outcomes. Extensive experiments across five fluid dynamics benchmarks (Prometheus, Navier-Stokes, Spherical-SWE, 3D Reaction-Diffusion, ERA5) demonstrate AGODE's state-of-the-art performance. Specifically, AGODE achieves MSE of 0.0302/0.0312 (in-domain/OOD) on Prometheus, outperforming PURE by 6.5%/5.0%, and reduces Navier-Stokes errors by 28.1% compared to physics-informed NMO. Notably, AGODE maintains superior OOD generalization with only 2.9% average error increase versus 7.8% for baselines, while its uncertainty quantification improves prediction reliability by 41% (95% confidence interval coverage). These results validate AGODE's capabilities in continuous spatiotemporal modeling, multi-parameter adaptation, and robust uncertainty estimation for complex fluid systems.
Timing is Important: Risk-aware Fund Allocation based on Time-Series Forecasting
Authors: Fuyuan Lyu, Linfeng Du, Yunpeng Weng, Qiufang Ying, Zhiyan Xu, Wen Zou, Haolun Wu, Xiuqiang He, Xing Tang
Venue: KDD
Year: 2025

Fund allocation has been an increasingly important problem in the financial domain. In reality, we aim to allocate the funds to buy certain assets within a certain future period. Naive solutions such as prediction-only or Predict-then-Optimize approaches suffer from goal mismatch. Additionally, the introduction of the SOTA time series forecasting model inevitably introduces additional uncertainty in the predicted result. To solve both problems mentioned above, we introduce a Risk-aware Time-Series Predict-and-Allocate (RTS-PnO) framework, which holds no prior assumption on the forecasting models. Such a framework contains three features: (i) end-to-end training with objective alignment measurement(ii) adaptive forecasting uncertainty calibration, and (iii) agnostic towards forecasting models. The evaluation of RTS-PnO is conducted over both online and offline experiments. For offline experiments, eight datasets from three categories of financial applications are used: Currency, Stock, and Cryptos. RTS-PnO consistently outperforms other competitive baselines. The online experiment is conducted on the Cross-Border Payment business at FiT, Tencent, and an 8.4% decrease in regret is witnessed when compared with the product-line approach. The code for the offline experiment is available.
NRFormer: Nationwide Nuclear Radiation Forecasting with Spatio-Temporal Transformer
Authors: Tengfei Lyu, Jindong Han, Hao Liu
Venue: KDD
Year: 2025

Nuclear radiation, which refers to the energy emitted from atomic nuclei during decay, poses significant risks to human health and environmental safety. Recently, advancements in monitoring technology have facilitated the effective recording of nuclear radiation levels and related factors, such as weather conditions. The abundance of monitoring data enables the development of accurate and reliable nuclear radiation forecasting models, which play a crucial role in informing decision-making for individuals and governments. However, this task is challenging due to the imbalanced distribution of monitoring stations over a wide spatial range and the non-stationary radiation variation patterns. In this study, we introduce NRFormer, a novel framework tailored for the nationwide prediction of nuclear radiation variations. By integrating a non-stationary temporal attention module, an imbalance-aware spatial attention module, and a radiation propagation prompting module, NRFormer collectively captures complex spatio-temporal dynamics of nuclear radiation. Extensive experiments on two real-world datasets demonstrate the superiority of our proposed framework against 11 baselines. NRFormer has been deployed online to provide 1-24-day nuclear radiation forecasts, empowering individuals and governments with timely, data-driven decisions for emergency response and public safety. Our framework is designed for general applicability and can be readily adapted for deployment in other regions. The deployed system is available at https://NRFormer.github.io and the dataset and code of the predictive model are available at https://github.com/usail-hkust/NRFormer.
DV365: Extremely Long User History Modeling at Instagram
Authors: Wenhan Lyu, Devashish Tyagi, Yihang Yang, Ziwei Li, Ajay Somani, Karthikeyan Shanmugasundaram, Nikola Andrejevic, Ferdi Adeputra, Curtis Zeng, Arun K. Singh, Maxime Ransan, Sagar Jain
Venue: KDD
Year: 2025

Long user history is highly valuable signal for recommendation systems, but effectively incorporating it often comes with high cost in terms of data center power consumption and GPU. In this work, we chose offline embedding over end-to-end sequence length optimization methods to enable extremely long user sequence modeling as a cost-effective solution, and propose a new user embedding learning strategy, multi-slicing and summarization, that generates highly generalizable user representation of user's long-term stable interest. History length we encoded in this embedding is up to 70,000 and on average 40,000. This embedding, named as DV365, is proven highly incremental on top of advanced attentive user sequence models deployed in Instagram. Produced by a single upstream foundational model, it is launched in 15 different models across Instagram and Threads with significant impact, and has been production battle-proven for >1 year since our first launch.
AutoSTF: Decoupled Neural Architecture Search for Cost-Effective Automated Spatio-Temporal Forecasting
Authors: Tengfei Lyu, Weijia Zhang, Jinliang Deng, Hao Liu
Venue: KDD
Year: 2025

Spatio-temporal forecasting is a critical component of various smart city applications, such as transportation optimization, energy management, and socio-economic analysis. Recently, several automated spatio-temporal forecasting methods have been proposed to automatically search the optimal neural network architecture for capturing complex spatio-temporal dependencies. However, the existing automated approaches suffer from expensive neural architecture search overhead, which hinders their practical use and the further exploration of diverse spatio-temporal operators in a finer granularity. In this paper, we propose AutoSTF, a decoupled automatic neural architecture search framework for cost-effective automated spatio-temporal forecasting. From the efficiency perspective, we first decouple the mixed search space into temporal space and spatial space and respectively devise representation compression and parameter-sharing schemes to mitigate the parameter explosion. The decoupled spatio-temporal search not only expedites the model optimization process but also leaves new room for more effective spatio-temporal dependency modeling. From the effectiveness perspective, we propose a multi-patch transfer module to jointly capture multi-granularity temporal dependencies and extend the spatial search space to enable finer-grained layer-wise spatial dependency search. Extensive experiments on eight datasets demonstrate the superiority of AutoSTF in terms of both accuracy and efficiency. Specifically, our proposed method achieves up to 13.48x speed-up compared to state-of-the-art automatic spatio-temporal forecasting methods while maintaining the best forecasting accuracy. The source code and data are available at https://github.com/usail-hkust/AutoSTF.
Task Diversity in Bayesian Federated Learning: Simultaneous Processing of Classification and Regression
Authors: Junliang Lyu, Yixuan Zhang, Xiaoling Lu, Feng Zhou
Venue: KDD
Year: 2025

This work addresses a key limitation in current federated learning approaches, which predominantly focus on homogeneous tasks, neglecting the task diversity on local devices. We propose a principled integration of multi-task learning using multi-output Gaussian processes (MOGP) at the local level and federated learning at the global level. MOGP handles correlated classification and regression tasks, offering a Bayesian non-parametric approach that naturally quantifies uncertainty. The central server aggregates the posteriors from local devices, updating a global MOGP prior redistributed for training local models until convergence. Challenges in performing posterior inference on local devices are addressed through the Polya-Gamma augmentation technique and mean-field variational inference, enhancing computational efficiency and convergence rate. Experimental results on both synthetic and real data demonstrate superior predictive performance, OOD detection, uncertainty calibration and convergence rate, highlighting the method's potential in diverse applications. Our code is publicly available at https://github.com/JunliangLv/task_diversity_BFL.
Achieving Nearly-Optimal Regret and Sample Complexity in Dueling Bandits with Applications in Online Recommendations
Authors: Lanjihong Ma, Yao-Xiang Ding, Zhen-Yu Zhang, Zhi-Hua Zhou
Venue: KDD
Year: 2025

We focus on the dueling bandits problem, which has recently drawn significant attention due to its wide-ranging applications in online recommendation systems and the alignment of large language models (LLMs), considers an online preference learning scenario where the learner iteratively selects arms based on pairwise comparison feedback to infer user preferences. Two primary objectives are typically considered in dueling bandits: Regret Minimization (RM), which aims to improve the overall quality of selected arms over time, and Best Arm Identification (BAI), which seeks to efficiently identify the best item with minimal user feedback. For instance, RM is exemplified by the objective of consistently providing high-quality items, while BAI reduces the required human feedback by minimizing the number of necessary comparisons. Conventional research treats RM and BAI as two conflicting objectives, optimizing one at the expense of the other. In this paper, we propose a novel framework that demonstrates the near-consistency of RM and BAI in dueling bandits by reducing the BAI in dueling bandits into a sequential noisy identification problem. Based on our formulation, we propose a black-box reduction technique that transforms any RM algorithm into a BAI algorithm, and prove that such reduction with optimal RM algorithm achieves optimal sample complexity and nearly-optimal cumulative weak regret simultaneously. Our proposed algorithm acheives a nearly-optimal BAI sample complexity and attains a cumulative weak regret that is order-wise equivalent to the best-known result simultaneously. Experiments on both synthetic benchmarks and real-world online recommendation tasks validate the effectiveness of the proposed method, providing empirical evidences for our theoretical findings.
ALSA: Context-Sensitive Prompt Privacy Preservation in Large Language Models
Authors: Hongru Ma, Wenpeng Lu, Yanjie Liang, Tianyi Wang, Qi Zhang, Yingjie Zhu, Jiasheng Si
Venue: KDD
Year: 2025

The remarkable prompting capability of large language models (LLMs) offers substantial convenience to users across diverse backgrounds. Nevertheless, as the sensitive information within prompts is inevitably exposed to LLMs, caution must be exercised to preserve privacy. Among various studies, text anonymization is considered an effective approach to preventing privacy leakage in prompts through text substitution. However, existing works overemphasize privacy while overlooks preserving contextual integrity, degrading semantic consistency. To address these concerns, this paper introduces a context-sensitive prompt privacy-preserving framework, namely Adaptive Linguistic Sanitization and Anonymization (ALSA). In specific, ALSA incorporates a three-dimensional scoring mechanism to dynamically quantify the substitutability of each word within a prompt by integrating the Privacy Leakage Risk Score (PLRS), the Contextual Information Importance Score (CIIS), and the Task Relevance Score (TRS). Subsequently, a clustering technique is adopted to dynamically determine the threshold for assigning an anonymization action (i.e., Retain, Replace, Encrypt, or Delete) by balancing privacy, semantics, and task relevance. Extensive experiments on five benchmark datasets validate the superiority of ALSA over state-of-the-art baselines in terms of accuracy, privacy preservation, and semantic integrity.
Beyond Fixed Variables: Expanding-variate Time Series Forecasting via Flat Scheme and Spatio-temporal Focal Learning
Authors: Minbo Ma, Kai Tang, Huan Li, Fei Teng, Dalin Zhang, Tianrui Li
Venue: KDD
Year: 2025

Multivariate Time Series Forecasting (MTSF) has long been a key research focus. Traditionally, these studies assume a fixed number of variables, but in real-world applications, Cyber-Physical Systems often expand as new sensors are deployed, increasing variables in MTSF. In light of this, we introduce a novel task, Expanding-variate Time Series Forecasting (EVTSF). This task presents unique challenges, specifically (1) handling inconsistent data shapes caused by adding new variables, and (2) addressing imbalanced spatio-temporal learning, where expanding variables have limited observed data due to the necessity for timely operation. To address these challenges, we propose STEV, a flexible spatio-temporal forecasting framework. STEV includes a new Flat Scheme to tackle the inconsistent data shape issue, which extends the graph-based spatio-temporal modeling architecture into 1D space by flattening the 2D samples along the variable dimension, making the model variable-scale-agnostic while still preserving dynamic spatial correlations through a holistic graph. Additionally, we introduce a novel Spatio-temporal Focal Learning strategy that incorporates a negative filter to resolve potential conflicts between contrastive learning and graph representation, and a focal contrastive loss as its core to guide the framework to focus on optimizing the expanding variables. To evaluate the effectiveness of STEV, we benchmark EVTSF performance on three real-world datasets from various domains and compare it against three potential solutions employing state-of-the-art (SOTA) MTSF models tailored for EVSTF. Experimental results show that STEV significantly outperforms its competitors, especially in handling expanding variables. Notably, STEV, with only 5% of observations during the expanding period, is on par with SOTA MTSF models trained with complete data. Further exploration of various expanding scenarios underscores the generalizability of STEV in real-world applications.
T3Set: A Multimodal Dataset with Targeted Suggestions for LLM-based Virtual Coach in Table Tennis Training
Authors: Ji Ma, Jiale Wu, Haoyu Wang, Yanze Zhang, Xiao Xie, Zheng Zhou, Hui Zhang, Jiachen Wang, Yingcai Wu
Venue: KDD
Year: 2025

Coaching is critical for learning table tennis skills. However, amateur table tennis players often lack access to professional coaches due to high costs and a limited number of coaches. While recent multimodal large language models show promise as virtual coaches, most of the existing approaches merely rely on video analysis, which is not comprehensive enough. In table tennis, many important kinematic details (e.g., strength, acceleration) cannot be captured by videos. They can only be tracked using sensors. To address this gap, we present T3Set (Table Tennis Training Set), a multimodal dataset that synchronizes inertial measurement unit (IMU) data from sensors mounted on 32 players' rackets with video recordings. The sensor data has 16 dimensions and a sample rate of 100Hz. This dataset covers 7 fundamental techniques across 380 training rounds, totaling 8655 annotated strokes, with 8395 targeted suggestions from coaches. The key features of T3Set include (1) temporal alignment between sensor data, video data, and text data. (2) high-quality targeted suggestions which are consistent with predefined suggestion taxonomy. Based on T3Set, we propose a novel two-stage framework that effectively integrates motion perception with generative reasoning as a virtual coach. Our method quantitatively outperforms baseline methods. The dataset, code, and documentation are available at https://github.com/jima-cs/T3Set.
Learning Patterns-Guided Data Generation for Knowledge Tracing
Authors: Haiping Ma, Yi Yin, Ziwen Wang, Changqian Wang, Xiaoshan Yu, Shangshang Yang, Xingyi Zhang
Venue: KDD
Year: 2025

Knowledge tracing (KT), which is instrumental in monitoring and forecasting students' knowledge states throughout their learning trajectory in online learning environments, has over the past decade garnered widespread attention due to its pivotal role in facilitating personalized education. Existing KT approaches were mainly invented from the model-centric perspective to overcome the sequence modeling difficulty while not exploiting the potential information of sparsity, thereby limiting their performance. To make full use of the information in the dataset, this paper proposes a data-centric knowledge tracing paradigm, termed LPDG, aiming to generate interaction data between students and exercises by revealing students' Learning Patterns and facilitating the Generation of ideal training Data. Specifically, we propose a learning patterns-guided exercise sequence regenerator, which incorporates Transformer and a tailor-made pattern enhancer, thereby aiding in the extraction of valuable information for generating high-quality training data. Moreover, we devise a learning patterns-guided pseudo-label generator, which leverages the diffusion process to construct pseudo-labels for the regenerated sequences. Afterwards, the fully generated ideal data is incorporated into the training data, and we integrate this framework with various model-centric approaches in KT. Finally, experimental results across datasets clearly demonstrate the efficacy of our proposed LPDG framework.
On the Support Vector Effect in DNNs: Rethinking Data Selection and Attribution
Authors: Syed Hasan Amin Mahmood, Ming Yin, Rajiv Khanna
Venue: KDD
Year: 2025

In Deep Neural Networks (DNNs), manipulating gradients is central to various algorithms, including data subset selection and instance attribution. For better tractability, practitioners often resort to using only the gradients of the last layer as a heuristic, instead of the full gradient across all model parameters, which we show is detrimental due to the Support Vector Effect (SVE). We introduce SVE, a max-margin-like behavior in the last layer(s) of DNNs and employ it to thoroughly scrutinize prevalent data selection and attribution methods relying on last layer gradients. Our investigation exposes limitations in these techniques and not only provides explanations for previously observed pitfalls, like lack of diversity and temporal performance degradation, but also offers fresh insights, including the vulnerability of existing methods to basic poisoning attacks and the potential for competitive performance using much simpler alternatives. Based on insights from SVE, we craft new methods RandE and PAE for data subset selection and instance attribution, respectively, which often outperform the purported state-of-the-art at a fraction of the cost, emphasizing the practical advantages of more efficient and less complex approaches.
SPARTA: An Optimization Framework for Differentially Private Sparse Fine-Tuning
Authors: Mehdi Makni, Kayhan Behdin, Gabriel Afriat, Zheng Xu, Sergei Vassilvitskii, Natalia Ponomareva, Rahul Mazumder, Hussein Hazimeh
Venue: KDD
Year: 2025

Differentially private stochastic gradient descent (DP-SGD) is broadly considered to be the gold standard for training and fine-tuning neural networks under differential privacy (DP). With the increasing availability of high-quality pre-trained model checkpoints (e.g., vision and language models), fine-tuning has become a popular strategy. However, despite recent progress in understanding and applying DP-SGD for private transfer learning tasks, significant challenges remain - most notably, the performance gap between models fine-tuned with DP-SGD and their non-private counterparts. Sparse fine-tuning on private data has emerged as an alternative to full-model fine-tuning -- recent work has shown that privately fine-tuning only a small subset of model weights and keeping the rest of the weights fixed can lead to better performance. In this work, we propose a new approach for sparse fine-tuning of neural networks under DP. Existing work on private sparse finetuning often used fixed choice of trainable weights (e.g., updating only the last layer), or relied on public model's weights to choose the subset of weights to modify. Such choice of weights remains suboptimal. In contrast, we explore an optimization-based approach, where our selection method makes use of the private gradient information, while using off the shelf privacy accounting techniques. Our numerical experiments on several computer vision models and datasets show that our parameter selection method leads to better prediction accuracy, compared to full-model private fine-tuning or existing private sparse fine-tuning approaches. Our code is available here: https://github.com/mazumder-lab/SPARTA/tree/main