OpenRaiser/Intern-Atlas huggingface dataset:
Flexible Graph Neural Diffusion with Latent Class Representation Learning
Authors: Liangtian Wan, Huijin Han, Lu Sun 0004, Zixun Zhang, Zhaolong Ning, Xiaoran Yan, Feng Xia 0001
Venue: KDD
Year: 2024

In existing graph data, the connection relationships often exhibit uniform weights, leading to the model aggregating neighboring nodes with equal weights across various connection types. However, this uniform aggregation of diverse information diminishes the discriminability of node representations, contributing significantly to the over-smoothing issue in models. In this paper, we propose the Flexible Graph Neural Diffusion (FGND) model, incorporating latent class representation to address the misalignment between graph topology and node features. In particular, we combine latent class representation learning with the inherent graph topology to reconstruct the diffusion matrix during the graph diffusion process. We introduce the sim metric to quantify the degree of mismatch between graph topology and node features. By flexibly adjusting the dependency level on node features through the hyperparameter, we accommodate diverse adjacency relationships. The effective filtering of noise in the topology also allows the model to capture higher order information, significantly alleviating the over-smoothing problem. Meanwhile, we model the graphical diffusion process as a set of differential equations and employ advanced partial differential equation tools to obtain more accurate solutions. Empirical evaluations on five benchmarks reveal that our FGND model outperforms existing popular GNN methods in terms of both overall performance and stability under data perturbations. Meanwhile, our model exhibits superior performance in comparison to models tailored for heterogeneous graphs and those designed to address oversmoothing issues.
Generative AI in E-Commerce: What Can We Expect?
Authors: Haixun Wang
Venue: KDD
Year: 2024

The impact of generative AI on e-commerce is profound. It has significantly improved the understanding of user intent and serves as a comprehensive product knowledge graph. However, the most substantial disruptions are yet to come, partic- ularly through the rise of autonomous agents. In this talk, I will outline a tentative path toward a future where e-commerce not only offers an unparalleled customer experience but also thrives in a world dominated by generative AI and autonomous agents.
Global Human-guided Counterfactual Explanations for Molecular Properties via Reinforcement Learning
Authors: Danqing Wang, Antonis Antoniades, Kha-Dinh Luong, Edwin Zhang, Mert Kosan, Jiachen Li, Ambuj K. Singh, William Yang Wang, Lei Li 0005
Venue: KDD
Year: 2024

Counterfactual explanations of Graph Neural Networks (GNNs) offer a powerful way to understand data that can naturally be represented by a graph structure. Furthermore, in many domains, it is highly desirable to derive data-driven global explanations or rules that can better explain the high-level properties of the models and data in question. However, evaluating global counterfactual explanations is hard in real-world datasets due to a lack of human-annotated ground truth, which limits their use in areas like molecular sciences. Additionally, the increasing scale of these datasets provides a challenge for random search-based methods. In this paper, we develop a novel global explanation model RLHEX for molecular property prediction. It aligns the counterfactual explanations with human-defined principles, making the explanations more interpretable and easy for experts to evaluate. RLHEX includes a VAE-based graph generator to generate global explanations and an adapter to adjust the latent representation space to human-defined principles. Optimized by Proximal Policy Optimization (PPO), the global explanations produced by RLHEX cover 4.12% more input graphs and reduce the distance between the counterfactual explanation set and the input set by 0.47% on average across three molecular datasets. RLHEX provides a flexible framework to incorporate different human-designed principles into the counterfactual explanation generation process, aligning these explanations with domain expertise. The code and data are released at https://github.com/dqwang122/RLHEX.
POND: Multi-Source Time Series Domain Adaptation with Information-Aware Prompt Tuning
Authors: Junxiang Wang, Guangji Bai, Wei Cheng 0002, Zhengzhang Chen, Liang Zhao 0002, Haifeng Chen
Venue: KDD
Year: 2024

Time series domain adaptation stands as a pivotal and intricate challenge with diverse applications, including but not limited to human activity recognition, sleep stage classification, and machine fault diagnosis. Despite the numerous domain adaptation techniques proposed to tackle this complex problem, they primarily focus on domain adaptation from a single source domain. Yet, it is more crucial to investigate domain adaptation from multiple domains due to the potential for greater improvements. To address this, three important challenges need to be overcome: 1). The lack of exploration to utilize domain-specific information for domain adaptation, 2). The difficulty to learn domain-specific information that changes over time, and 3). The difficulty to evaluate learned domain-specific information. In order to tackle these challenges simultaneously, in this paper, we introduce PrOmpt-based domaiN Discrimination (POND), the first framework to utilize prompts for time series domain adaptation. Specifically, to address Challenge 1, we extend the idea of prompt tuning to time series analysis and learn prompts to capture common and domain-specific information from all source domains. To handle Challenge 2, we introduce a conditional module for each source domain to generate prompts from time series input data. For Challenge 3, we propose two criteria to select good prompts, which are used to choose the most suitable source domain for domain adaptation. The efficacy and robustness of our proposed POND model are extensively validated through experiments across 50 scenarios encompassing four datasets. Experimental results demonstrate that our proposed POND model outperforms all state-of-the-art comparison methods by up to 66% on the F1-score.
KDD 2024 Finance Day
Authors: Guiling Wang 0001, Daniel Borrajo
Venue: KDD
Year: 2024

The Finance Day at KDD 2024 will take place on August 26th in Barcelona, Spain. Following the success of the inaugural event last year, the second edition highlights the significant role of AI in transforming the financial industry. This special day serves as a forum for discussion of innovations at the intersection of AI and finance. An exciting lineup of 12 influential speakers from nine different countries will be featured, representing a mix of government organizations, leading banks, innovative hedge funds, and top academic institutions. These experts will delve into a range of topics, from cutting-edge FinTech innovations to ethical considerations in machine learning, providing a comprehensive overview of the finance and AI. The distinguished speakers include Avanidhar Subrahmanyam from UCLA, Henrike Mueller from the Financial Conduct Authority, Claudia Perlich from Two Sigma, Eyke Hüllermeier from Ludwig-Maximilians-Universität München, Senthil Kumar from Capital One, Stefan Zohren from the University of Oxford, Dumitru Roman from SINTEF ICT, Kubilay Atasu from TU Delft, Xiao-Ming Wu from Hong Kong Polytechnic University, Yongjae Lee from UNIST, Jundong Li from the University of Virginia, and Milos Blagojevic from BlackRock.
Multi-Scale Detection of Anomalous Spatio-Temporal Trajectories in Evolving Trajectory Datasets
Authors: Chenhao Wang 0007, Lisi Chen 0001, Shuo Shang, Christian S. Jensen, Panos Kalnis
Venue: KDD
Year: 2024

A trajectory is a sequence of timestamped point locations that captures the movement of an object such as a vehicle. Such trajectories encode complex spatial and temporal patterns and provide rich information about object mobility and the underlying infrastructures, typically road networks, within which the movements occur. A trajectory dataset is evolving when new trajectories are included continuously. The ability to detect anomalous trajectories in online fashion in this setting is fundamental and challenging functionality that has many applications, e.g., location-based services. State-of-the-art solutions determine anomalies based on the shapes or routes of trajectories, ignoring potential anomalies caused by different sampling rates or time offsets. We propose a multi-scale model, termed MST-OATD, for anomalous streaming trajectory detection that considers both the spatial and temporal aspects of trajectories. The model's multi-scale capabilities aim to enable extraction of trajectory features at multiple scales. In addition, to improve model evolvability and to contend with changes in trajectory patterns, the model is equipped with a learned ranking model that updates the training set as new trajectories are included. Experiments on real datasets offer evidence that the model can outperform state-of-the-art solutions and is capable of real-time anomaly detection. Further, the learned ranking model achieves promising results when updating the training set with newly arrived trajectories.
Pre-Training with Transferable Attention for Addressing Market Shifts in Cross-Market Sequential Recommendation
Authors: Chen Wang 0052, Ziwei Fan 0001, Liangwei Yang, Mingdai Yang, Xiaolong Liu 0012, Zhiwei Liu 0001, Philip S. Yu
Venue: KDD
Year: 2024

Cross-market recommendation (CMR) involves selling the same set of items across multiple nations or regions within a transfer learning framework. However, CMR's distinctive characteristics, including limited data sharing due to privacy policies, absence of user overlap, and a shared item set between markets present challenges for traditional recommendation methods. Moreover, CMR experiences market shifts, leading to differences in item popularity and user preferences among different markets. This study focuses on cross-market sequential recommendation (CMSR) and proposes the Cross-market Attention Transferring with Sequential Recommendation (CAT-SR) framework to address these challenges and market shifts. CAT-SR incorporates a pre-training strategy emphasizing item-item correlation, selective self-attention transferring for effective transfer learning, and query and key adapters for market-specific user preferences. Experimental results on real-world cross-market datasets demonstrate the superiority of CAT-SR, and ablation studies validate the benefits of its components across different geographical continents. CAT-SR offers a robust and adaptable solution for cross-market sequential recommendation. The code is available at https://github.com/ChenMetanoia/CATSR-KDD/.
Know in AdVance: Linear-Complexity Forecasting of Ad Campaign Performance with Evolving User Interest
Authors: Xiaoyu Wang 0014, Yonghui Guo, Hui Sheng, Peili Lv, Chi Zhou, Wei Huang, Shiqin Ta, Dongbo Huang, Xiujin Yang, Lan Xu 0001, Hao Zhou 0001, Yusheng Ji
Venue: KDD
Year: 2024

Real-time Bidding (RTB) advertisers wish to know in advance the expected cost and yield of ad campaigns to avoid trial-and-error expenses. However, Campaign Performance Forecasting (CPF), a sequence modeling task involving tens of thousands of ad auctions, poses challenges of evolving user interest, auction representation, and long context, making coarse-grained and static-modeling methods sub-optimal. We propose AdVance, a time-aware framework that integrates local auction-level and global campaign-level modeling. User preference and fatigue are disentangled using a time-positioned sequence of clicked items and a concise vector of all displayed items. Cross-attention, conditioned on the fatigue vector, captures the dynamics of user interest toward each candidate ad. Bidders compete with each other, presenting a complete graph similar to the self-attention mechanism. Hence, we employ a Transformer Encoder to compress each auction into embedding by solving auxiliary tasks. These sequential embeddings are then summarized by a conditional state space model (SSM) to comprehend long-range dependencies while maintaining global linear complexity. Considering the irregular time intervals between auctions, we make SSM's parameters dependent on the current auction embedding and the time interval. We further condition SSM's global predictions on the accumulation of local results. Extensive evaluations and ablation studies demonstrate its superiority over state-of-the-art methods. AdVance has been deployed on the Tencent Advertising platform, and A/B tests show a remarkable 4.5% uplift in Average Revenue per User (ARPU).
AsyncET: Asynchronous Representation Learning for Knowledge Graph Entity Typing
Authors: Yun-Cheng Wang, Xiou Ge, Bin Wang 0040, C.-C. Jay Kuo
Venue: KDD
Year: 2024

Knowledge graph entity typing (KGET) aims to predict the missing entity types in knowledge graphs (KG). The relationship between entities and their corresponding types is often expressed using a single relation, hasType. However, hasType has a limited capability for modeling diverse entity-type relationships in the embedding space. In this paper, we first introduce multiple auxiliary relations to model the complex entity-type relationship. We propose an efficient and robust algorithm to group similar entity types together and assign a unique auxiliary relation to each group. Then, with the auxiliary relations, we propose an Asynchronous representation learning framework for KGET, named AsyncET, where entity and type embeddings are updated alternatively. Consequently, the quality of entity embeddings is gradually improved during training by infusing type information. In addition, entity types with different granularities and semantics can be properly modeled in the embedding space. Experimental results show that AsyncET can substantially improve the performance of embedding-based methods on the KGET task and has a significant advantage over state-of-the-art neural network-based methods in terms of model sizes and inference time.
BacktrackSTL: Ultra-Fast Online Seasonal-Trend Decomposition with Backtrack Technique
Authors: Haoyu Wang, Hongke Guo, Zhaoliang Zhu, You Zhang, Yu Zhou, Xudong Zheng
Venue: KDD
Year: 2024

Seasonal-trend decomposition (STD) is a crucial task in time series data analysis. Due to the challenges of scalability, there is a pressing need for an ultra-fast online algorithm. However, existing algorithms either fail to handle long-period time series (such as OnlineSTL), or need time-consuming iterative processes (such as OneShotSTL). Therefore, we propose BacktrackSTL, the first non-iterative online STD algorithm with period-independent O(1) update complexity. It is also robust to outlier, seasonality shift and trend jump because of the combination of outlier-resilient smoothing, non-local seasonal filtering and backtrack technique. Experimentally, BacktrackSTL decomposes a value within 1.6 μs, which is 15X faster than the state-of-the-art online algorithm OneShotSTL, while maintaining comparable accuracy to the best offline algorithm RobustSTL. We have also deployed BacktrackSTL on the top of Apache Flink to decompose monitoring metrics in Alibaba Cloud for over a year. Besides, we have open-sourced the artifact of this proposal on GitHub.
DyPS: Dynamic Parameter Sharing in Multi-Agent Reinforcement Learning for Spatio-Temporal Resource Allocation
Authors: Jingwei Wang 0002, Qianyue Hao, Wenzhen Huang, Xiaochen Fan, Zhentao Tang, Bin Wang 0034, Jianye Hao, Yong Li 0008
Venue: KDD
Year: 2024

In large-scale metropolis, it is critical to efficiently allocate various resources such as electricity, medical care, and transportation to meet the living demands of citizens, according to the spatio-temporal distributions of resources and demands. Previous researchers have done plentiful work on such problems by leveraging Multi-Agent Reinforcement Learning (MARL) methods, where multiple agents cooperatively regulate and allocate the resources to meet the demands. However, facing the great number of agents in large cities, existing MARL methods lack efficient parameter sharing strategies among agents to reduce computational complexity. There remain two primary challenges in efficient parameter sharing: (1) during the RL training process, the behavior of agents changes significantly, limiting the performance of group parameter sharing based on fixed role division decided before training; (2) the behavior of agents forms complicated action trajectories, where their role characteristics are implicit, adding difficulty to dynamically adjusting agent role divisions during the training process. In this paper, we propose Dynamic Parameter Sharing (DyPS) to solve the above challenges. We design self-supervised learning tasks to extract the implicit behavioral characteristics from the action trajectories of agents. Based on the obtained behavioral characteristics, we propose a hierarchical MARL framework capable of dynamically revising the agent role divisions during the training process and thus shares parameters among agents with the same role, reducing computational complexity. In addition, our framework can be combined with various typical MARL algorithms, including IPPO, MAPPO, etc. We conduct 7 experiments in 4 representative resource allocation scenarios, where extensive results demonstrate our method's superior performance, outperforming the state-of-the-art baseline methods by up to 31%. Our source codes are available at https://github.com/tsinghua-fib-lab/DyPS.
Robust Predictions with Ambiguous Time Delays: A Bootstrap Strategy
Authors: Jiajie Wang, Zhiyuan Jerry Lin, Wen Chen
Venue: KDD
Year: 2024

In contemporary data-driven environments, the generation and processing of multivariate time series data is an omnipresent challenge, often complicated by time delays between different time series. These delays, originating from a multitude of sources like varying data transmission dynamics, sensor interferences, and environmental changes, introduce significant complexities. Traditional Time Delay Estimation methods, which typically assume a fixed constant time delay, may not fully capture these variabilities, compromising the precision of predictive models in diverse settings. To address this issue, we introduce the Time Series Model Bootstrap (TSMB), a versatile framework designed to handle potentially varying or even nondeterministic time delays in time series modeling. Contrary to traditional approaches that hinge on the assumption of a single, consistent time delay, TSMB adopts a nonparametric stance, acknowledging and incorporating time delay uncertainties. TSMB significantly bolsters the performance of models that are trained and make predictions using this framework, making it highly suitable for a wide range of dynamic and interconnected data environments.
Warming Up Cold-Start CTR Prediction by Learning Item-Specific Feature Interactions
Authors: Yaqing Wang 0002, Hongming Piao, Daxiang Dong, Quanming Yao, Jingbo Zhou 0003
Venue: KDD
Year: 2024

In recommendation systems, new items are continuously introduced, initially lacking interaction records but gradually accumulating them over time. Accurately predicting the click-through rate (CTR) for these items is crucial for enhancing both revenue and user experience. While existing methods focus on enhancing item ID embeddings for new items within general CTR models, they tend to adopt a global feature interaction approach, often overshadowing new items with sparse data by those with abundant interactions. Addressing this, our work introduces EmerG, a novel approach that warms up cold-start CTR prediction by learning item-specific feature interaction patterns. EmerG utilizes hypernetworks to generate an item-specific feature graph based on item characteristics, which is then processed by a Graph Neural Network (GNN). This GNN is specially tailored to provably capture feature interactions at any order through a customized message passing mechanism. We further design a meta learning strategy that optimizes parameters of hypernetworks and GNN across various item CTR prediction tasks, while only adjusting a minimal set of item-specific parameters within each task. This strategy effectively reduces the risk of overfitting when dealing with limited data. Extensive experiments on benchmark datasets validate that EmerG consistently performs the best given no, a few and sufficient instances of new items.
LiMAML: Personalization of Deep Recommender Models via Meta Learning
Authors: Ruofan Wang, Prakruthi Prabhakar, Gaurav Srivastava 0001, Tianqi Wang, Zeinab S. Jalali, Varun Bharill, Yunbo Ouyang, Aastha Nigam, Divya Venugopalan, Aman Gupta, Fedor Borisyuk, S. Sathiya Keerthi, Ajith Muralidharan
Venue: KDD
Year: 2024

In the realm of recommender systems, the ubiquitous adoption of deep neural networks has emerged as a dominant paradigm for modeling diverse business objectives. As user bases continue to expand, the necessity of personalization and frequent model updates have assumed paramount significance to ensure the delivery of relevant and refreshed experiences to a diverse array of members. In this work, we introduce an innovative meta-learning solution tailored to the personalization of models for individual members and other entities, coupled with the frequent updates based on the latest user interaction signals. Specifically, we leverage the Model-Agnostic Meta Learning (MAML) algorithm to adapt per-task sub-networks using recent user interaction data. Given the near infeasibility of productionizing original MAML-based models in online recommendation systems, we propose an efficient strategy to operationalize meta-learned sub-networks in production, which involves transforming them into fixed-sized vectors, termed meta embeddings, thereby enabling the seamless deployment of models with hundreds of billions of parameters for online serving. Through extensive experimentation on production data drawn from various applications at LinkedIn, we demonstrate that the proposed solution consistently outperforms the best performing baseline models of those applications, including strong baselines such as using wide-and-deep ID based personalization approach. Our approach has enabled the deployment of a range of highly personalized AI models across diverse LinkedIn applications, leading to substantial improvements in business metrics as well as refreshed experience for our members.
Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustness
Authors: Dingrong Wang, Hitesh Sapkota, Zhiqiang Tao, Qi Yu 0001
Venue: KDD
Year: 2024

Prior research on neural architecture search (NAS) for adversarial robustness has revealed that a lightweight and adversarially robust sub-network could exist in a non-robust large teacher network. Such a sub-network is generally discovered based on heuristic rules to perform neural architecture search. However, heuristic rules are inadequate to handle diverse adversarial attacks and different "teacher" network capacity. To address this key challenge, we propose Reinforced Compressive Neural Architecture Search (RC-NAS), aiming to achieve Versatile Adversarial Robustness. Specifically, we define novel task settings that compose datasets, adversarial attacks, and teacher network configuration. Given diverse tasks, we develop an innovative dual-level training paradigm that consists of a meta-training and a fine-tuning phase to effectively expose the RL agent to diverse attack scenarios (in meta-training), and make it adapt quickly to locate an optimal sub-network (in fine-tuning) for previously unseen scenarios. Experiments show that our framework could achieve adaptive compression towards different initial teacher networks, datasets, and adversarial attacks, resulting in more lightweight and adversarially robust architectures. We also provide a theoretical analysis to explain why the reinforcement learning (RL)-guided adversarial architectural search helps adversarial robustness over standard adversarial training methods.