OpenRaiser/Intern-Atlas huggingface dataset:
WHEN: A Wavelet-DTW Hybrid Attention Network for Heterogeneous Time Series Analysis
Authors: Jingyuan Wang 0001, Chen Yang 0024, Xiaohan Jiang, Junjie Wu 0002
Venue: KDD
Year: 2023

Given its broad applications, time series analysis has gained substantial research attention but remains a very challenging task. Recent years have witnessed the great success of deep learning methods, eg., CNN and RNN, in time series classification and forecasting, but heterogeneity as the very nature of time series has not yet been addressed adequately and remains the performance "treadstone." In this light, we argue that the intra-sequence non-stationarity and inter-sequence asynchronism are two types of heterogeneities widely existed in multiple times series, and propose a hybrid attention network called WHEN as deep learning solution. WHEN features in two attention mechanisms in two different modules. In the WaveAtt module, we propose a novel data-dependent wavelet function and integrate it into the BiLSTM network as the wavelet attention, for the purpose of analyzing dynamic frequency components in nonstationary time series. In the DTWAtt module, we transform the dynamic time warping (DTW) technique into the form as the DTW attention, where all input sequences are synchronized with a universal parameter sequence to overcome the time distortion problem in multiple time series. WHEN with the hybrid attentions is then formulated as task-dependent neural network for either classification or forecasting tasks. Extensive experiments on 30 UEA datasets and 3 real-world datasets with rich competitive baselines demonstrate the excellent performance of our model. The ability of WHEN in dealing with time series heterogeneities is also elaborately explored via specially designed analysis.
An Empirical Study of Selection Bias in Pinterest Ads Retrieval
Authors: Yuan Wang 0076, Peifeng Yin, Zhiqiang Tao, Hari Venkatesan, Jin Lai, Yi Fang 0008, PJ Xiao
Venue: KDD
Year: 2023

Data selection bias has been a long-lasting challenge in the machine learning domain, especially in multi-stage recommendation systems, where the distribution of labeled items for model training is very different from that of the actual candidates during inference time. This distribution shift is even more prominent in the context of online advertising where the user base is diverse and the platform contains a wide range of contents. In this paper, we first investigate the data selection bias in the upper funnel (Ads Retrieval) of Pinterest's multi-cascade ads ranking system. We then conduct comprehensive experiments to assess the performance of various state-of-the-art methods, including transfer learning, adversarial learning, and unsupervised domain adaptation. Moreover, we further introduce some modifications into the unsupervised domain adaptation and evaluate the performance of different variants of this modified method. Our online A/B experiments show that the modified version of unsupervised domain adaptation (MUDA) could provide the largest improvements to the performance of Pinterest's advertisement ranking system compared with other methods and the one used in current production.
Root Cause Analysis for Microservice Systems via Hierarchical Reinforcement Learning from Human Feedback
Authors: Lu Wang 0029, Chaoyun Zhang, Ruomeng Ding, Yong Xu 0010, Qihang Chen, Wentao Zou, Qingjun Chen, Meng Zhang 0025, Xuedong Gao, Hao Fan, Saravan Rajmohan, Qingwei Lin, Dongmei Zhang 0001
Venue: KDD
Year: 2023

In microservice systems, the identification of root causes of anomalies is imperative for service reliability and business impact. This process is typically divided into two phases: (i)constructing a service dependency graph that outlines the sequence and structure of system components that are invoked, and (ii) localizing the root cause components using the graph, traces, logs, and Key Performance Indicators (KPIs) such as latency. However, both phases are not straightforward due to the highly dynamic and complex nature of the system, particularly in large-scale commercial architectures like Microsoft Exchange.
Removing Camouflage and Revealing Collusion: Leveraging Gang-crime Pattern in Fraudster Detection
Authors: Lewen Wang, Haozhe Zhao, Cunguang Feng, Weiqing Liu, Congrui Huang, Marco Santoni, Manuel Cristofaro, Paola Jafrancesco, Jiang Bian 0002
Venue: KDD
Year: 2023

As one of the major threats to the healthy development of various online platforms, fraud has become increasingly committed in the form of gangs since collusive fraudulent activities are much easier to obtain illicit benefits with lower exposure risk. To detect fraudsters in a gang, spatio-temporal graph neural network models have been widely applied to detect both temporal and spatial collusive patterns. However, a closer peek into real-world records of fraudsters can reveal that fraud gangs usually conduct community-level camouflage, specified by two types, i.e., temporal and spatial camouflage. Such camouflage can disguise gangs as benign communities by concealing collusive patterns and thus deceiving many existing graph neural network models. In the meantime, many existing graph neural network models suffer from the challenge of extreme sample imbalance caused by rare fraudsters hidden among massive users. To handle all these challenges, in this paper, we propose a generative adversarial network framework, named Adversarial Camouflage Detector, to detect fraudsters. Concretely, this ACD framework consists of four modules, in charge of community division, camouflage identification, fraudster detection, and camouflage generation, respectively. The first three modules form up a discriminator that uses spatio-temporal graph neural networks as the foundation model and enhance fraudster detection by amplifying the gangs' collusive patterns through automatically identifying and removing camouflage. Meanwhile, the camouflage generation module plays as the generator role that generates fraudsters samples by competing against the discriminator to alleviate the challenge of sample imbalance and increase the model robustness. The experimental result shows that our proposed method outperforms other methods on real-world datasets.
Granger Causal Chain Discovery for Sepsis-Associated Derangements via Continuous-Time Hawkes Processes
Authors: Song Wei, Yao Xie 0002, Christopher S. Josef, Rishikesan Kamaleswaran
Venue: KDD
Year: 2023

Modern health care systems are conducting continuous, automated surveillance of the electronic medical record (EMR) to identify adverse events with increasing frequency; however, many events such as sepsis do not have elucidated prodromes (i.e., event chains) that can be used to identify and intercept the adverse event early in its course. Clinically relevant and interpretable results require a framework that can (i) infer temporal interactions across multiple patient features found in EMR data (e.g., Labs, vital signs, etc.) and (ii) identify patterns that precede and are specific to an impending adverse event (e.g., sepsis). In this work, we propose a linear multivariate Hawkes process model, coupled with ReLU link function, to recover a Granger Causal (GC) graph with both exciting and inhibiting effects. We develop a scalable two-phase gradient-based method to obtain a maximum surrogate-likelihood estimator, which is shown to be effective via extensive numerical simulation. Our method is subsequently extended to a data set of patients admitted to Grady hospital system in Atlanta, GA, USA, where the estimated GC graph identifies several highly interpretable GC chains that precede sepsis. The code is available at https://github.com/SongWei-GT/two-phase-MHP.
To Aggregate or Not? Learning with Separate Noisy Labels
Authors: Jiaheng Wei, Zhaowei Zhu, Tianyi Luo, Ehsan Amid, Abhishek Kumar, Yang Liu 0018
Venue: KDD
Year: 2023

The rawly collected training data often comes with separate noisy labels collected from multiple imperfect annotators (e.g., via crowdsourcing). A typical way of using these separate labels is to first aggregate them into one and apply standard training methods. The literature has also studied extensively on effective aggregation approaches. This paper revisits this choice and aims to provide an answer to the question of whether one should aggregate separate noisy labels into single ones or use them separately as given. We theoretically analyze the performance of both approaches under the empirical risk minimization framework for a number of popular loss functions, including the ones designed specifically for the problem of learning with noisy labels. Our theorems conclude that label separation is preferred over label aggregation when the noise rates are high, or the number of labelers/annotations is insufficient. Extensive empirical results validate our conclusions.
Graph Neural Networks: Foundation, Frontiers and Applications
Authors: Lingfei Wu 0001, Peng Cui 0001, Jian Pei 0001, Liang Zhao 0002, Xiaojie Guo 0002
Venue: KDD
Year: 2023

The field of graph neural networks (GNNs) has seen rapid and incredible strides over the recent years. Graph neural networks, also known as deep learning on graphs, graph representation learning, or geometric deep learning, have become one of the fastest-growing research topics in machine learning, especially deep learning. However, as the field rapidly grows, it has been extremely challenging to gain a global perspective of the developments of GNNs. Therefore, we feel the urgency to bridge the above gap and have a comprehensive tutorial on this fast-growing yet challenging topic.
TransformerLight: A Novel Sequence Modeling Based Traffic Signaling Mechanism via Gated Transformer
Authors: Qiang Wu 0010, Mingyuan Li 0006, Jun Shen 0001, Linyuan Lü, Bo Du 0004, Ke Zhang
Venue: KDD
Year: 2023

Traffic signal control (TSC) is still one of the most significant and challenging research problems in the transportation field. Reinforcement learning (RL) has achieved great success in TSC but suffers from critically high learning costs in practical applications due to the excessive trial-and-error learning process. Offline RL is a promising method to reduce learning costs whereas the data distribution shift issue is still up in the air. To this end, in this paper, we formulate TSC as a sequence modeling problem with a sequence of Markov decision process described by states, actions, and rewards from the traffic environment. A novel framework, namely TransformerLight, is introduced, which does not aim to fit into value functions by averaging all possible returns, but produces the best possible actions using a gated Transformer. Additionally, the learning process of TransformerLight is much more stable by replacing the residual connections with gated transformer blocks due to a dynamic system perspective. Through numerical experiments on offline datasets, we demonstrate that the TransformerLight model: (1) can build a high-performance adaptive TSC model without dynamic programming; (2) achieves a new state-of-the-art compared to most published offline RL methods so far; and (3) shows a more stable learning process than offline RL and recent Transformer-based methods. The relevant dataset and code are available at Github.
Deep Bayesian Active Learning for Accelerating Stochastic Simulation
Authors: Dongxia Wu, Ruijia Niu, Matteo Chinazzi, Alessandro Vespignani, Yi-An Ma, Rose Yu
Venue: KDD
Year: 2023

Stochastic simulations such as large-scale, spatiotemporal, age-structured epidemic models are computationally expensive at fine-grained resolution. While deep surrogate models can speed up the simulations, doing so for stochastic simulations and with active learning approaches is an underexplored area. We propose Interactive Neural Process (INP), a deep Bayesian active learning framework for learning deep surrogate models to accelerate stochastic simulations. INP consists of two components, a spatiotemporal surrogate model built upon Neural Process (NP) family and an acquisition function for active learning. For surrogate modeling, we develop Spatiotemporal Neural Process (STNP) to mimic the simulator dynamics. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. We perform a theoretical analysis and demonstrate that LIG reduces sample complexity compared with random sampling in high dimensions. We also conduct empirical studies on three complex spatiotemporal simulators for reaction diffusion, heat flow, and infectious disease. The results demonstrate that STNP outperforms the baselines in the offline learning setting and LIG achieves the state-of-the-art for Bayesian active learning.
Deep Learning on Graphs: Methods and Applications (DLG-KDD2023)
Authors: Lingfei Wu 0001, Jian Pei 0001, Jiliang Tang, Yinglong Xia, Xiaojie Guo 0002
Venue: KDD
Year: 2023

Deep Learning models are at the core of research in Artificial Intelligence research today. A tide in research for deep learning on graphs or graph neural networks. This wave of research at the intersection of graph theory and deep learning has also influenced other fields of science, including computer vision, natural language processing, program synthesis and analysis, financial security, Drug Discovery and so on. However, there are still many challenges regarding a broad range of the topics in deep learning on graphs, from methodologies to applications, and from foundations to the new frontiers of GNNs. This international workshop on "Deep Learning on Graphs: Method and Applications (DLG-KDD'23)" aims to bring together both academic researchers and industrial practitioners from different backgrounds and perspectives to above challenges.
Certified Edge Unlearning for Graph Neural Networks
Authors: Kun Wu 0011, Jie Shen 0005, Yue Ning 0001, Ting Wang 0006, Wendy Hui Wang
Venue: KDD
Year: 2023

The emergence of evolving data privacy policies and regulations has sparked a growing interest in the concept of "machine unlearning", which involves enabling machine learning models to forget specific data instances. In this paper, we specifically focus on edge unlearning in Graph Neural Networks (GNNs), which entails training a new GNN model as if certain specified edges never existed in the original training graph. Unlike conventional unlearning scenarios where data samples are treated as independent entities, edges in graphs exhibit correlation. Failing to carefully account for this data dependency would result in the incomplete removal of the requested data from the model. While retraining the model from scratch by excluding the specific edges can eliminate their influence, this approach incurs a high computational cost. To overcome this challenge, we introduce CEU, a Certified Edge Unlearning framework. CEU expedites the unlearning process by updating the parameters of the pre-trained GNN model in a single step, ensuring that the update removes the influence of the removed edges from the model. We formally prove that CEU offers a rigorous theoretical guarantee under the assumption of convexity on the loss function. Our empirical analysis further demonstrates the effectiveness and efficiency of CEU for both linear and deep GNNs - it achieves significant speedup gains compared to retraining and existing unlearning methods while maintaining comparable model accuracy to retraining from scratch.
Self-Adaptive Perturbation Radii for Adversarial Training
Authors: Huimin Wu 0004, Wanli Shi, Chenkang Zhang, Bin Gu 0001
Venue: KDD
Year: 2023

Adversarial training has been shown to be the most popular and effective technique to protect models from imperceptible adversarial samples. Despite its success, it also accompanies the significant performance degeneration to clean data. To achieve a good performance on both clean and adversarial samples, the main effort is searching for an adaptive perturbation radius for each training sample. However, this method suffers from a conflict between exact searching and computational overhead. To address this conflict, in this paper, firstly we show the superiority of adaptive perturbation radii on the accuracy and robustness respectively. Then we propose our novel self-adaptive adjustment framework for perturbation radii without tedious searching. We also discuss this framework on both deep neural networks (DNNs) and kernel support vector machines (SVMs). Finally, extensive experimental results show that our framework can improve adversarial robustness without compromising the natural generalization. It is also competitive with existing searching strategies in terms of running time.
Graph Contrastive Learning with Generative Adversarial Network
Authors: Cheng Wu 0004, Chaokun Wang, Jingcao Xu, Ziyang Liu 0004, Kai Zheng 0001, Xiaowei Wang, Yang Song 0008, Kun Gai
Venue: KDD
Year: 2023

Graph Neural Networks (GNNs) have demonstrated promising results on exploiting node representations for many downstream tasks through supervised end-to-end training. To deal with the widespread label scarcity issue in real-world applications, Graph Contrastive Learning (GCL) is leveraged to train GNNs with limited or even no labels by maximizing the mutual information between nodes in its augmented views generated from the original graph. However, the distribution of graphs remains unconsidered in view generation, resulting in the ignorance of unseen edges in most existing literature, which is empirically shown to be able to improve GCL's performance in our experiments. To this end, we propose to incorporate graph generative adversarial networks (GANs) to learn the distribution of views for GCL, in order to i) automatically capture the characteristic of graphs for augmentations, and ii) jointly train the graph GAN model and the GCL model. Specifically, we present GACN, a novel Generative Adversarial Contrastive learning Network for graph representation learning. GACN develops a view generator and a view discriminator to generate augmented views automatically in an adversarial style. Then, GACN leverages these views to train a GNN encoder with two carefully designed self-supervised learning losses, including the graph contrastive loss and the Bayesian personalized ranking Loss. Furthermore, we design an optimization framework to train all GACN modules jointly. Extensive experiments on seven real-world datasets show that GACN is able to generate high-quality augmented views for GCL and is superior to twelve state-of-the-art baseline methods. Noticeably, our proposed GACN surprisingly discovers that the generated views in data augmentation finally conform to the well-known preferential attachment rule in online networks.
A Predict-Then-Optimize Couriers Allocation Framework for Emergency Last-mile Logistics
Authors: Kaiwen Xia, Li Lin 0011, Shuai Wang 0008, Haotian Wang 0008, Desheng Zhang 0002, Tian He 0001
Venue: KDD
Year: 2023

In recent years, emergency last-mile logistics (ELML) have played an essential role in urban emergencies. The efficient allocation of couriers in ELML is of practical significance to ensure the supply of essential materials, especially in public health emergencies (PHEs). However, couriers allocation becomes challenging due to the instability of demand, dynamic supply comprehension, and the evolutional delivery environment for ELML caused by PHEs. While existing work has delved into couriers allocation, the impact of PHEs on demand-supply-delivery has yet to be considered. In this work, we design PTOCA, a Predict-Then-Optimize Couriers Allocation framework. Specifically, in the prediction stage, we design a resource-aware prediction module that performs spatio-temporal modeling of unstable demand characteristics using a variational graph GRU encoder and builds a task-resource regressor to predict demand accurately. In the optimization stage, firstly, the priority ranking module solves the matching of delivery resources under demand-supply imbalance. Then the multi-factor task allocation module is used to model the dynamic evolutional environment and reasonably assign the delivery tasks of couriers. We evaluate PTOCA using real-world data covering 170 delivery zones, more than 10,000 couriers, and 100 million delivery tasks. The data is collected from JD Logistics, one of the largest logistics service companies. Extensive experimental results show that our method outperforms the baseline in task delivery rate and on-time delivery rate.
User-Regulation Deconfounded Conversational Recommender System with Bandit Feedback
Authors: Yu Xia 0007, Junda Wu, Tong Yu 0001, Sungchul Kim, Ryan A. Rossi, Shuai Li 0010
Venue: KDD
Year: 2023

Recent conversational recommender systems (CRSs) have achieved considerable success on addressing the cold-start problem. While they utilize conversational key-terms to efficiently elicit user preferences, most of them, however, neglect that key-terms can also introduce biases. Systems learning key-term-level user preferences may make a biased item recommendation based on an overrated key-term instead of the item itself. As key-term conversation is a crucial part of CRSs, it is important to properly handle such bias resulting from the item-key-term relationship. While many debiasing methods have been proposed for traditional recommender systems, most of them focus on items or item groups re-ranking or re-weighting strategies such as calibration and propensity score, which are not designed to model the relation between item and key-term user preference. There is also no effective way for traditional debiasing methods to measure potentially useful biases through conversational key-terms to enhance the recommendation performance.
Knowledge Based Prohibited Item Detection on Heterogeneous Risk Graphs
Authors: Tingyan Xiang, Ao Li 0005, Yugang Ji, Dong Li 0037
Venue: KDD
Year: 2023

With the popularity of online shopping in recent years, various prohibited items are continuously attacking e-commerce portals. Searching and deleting such risk items online has played a fundamental role in protecting the health of e-commerce trades. To mitigate negative impact of limited supervision and adversarial behaviors of malicious sellers, current state-of-the-art work mainly introduces heterogeneous graph neural network with further improvements such as graph structure learning, pairwise training mechanism, etc. However, performance of these models is highly limited since domain knowledge is indispensable for identifying prohibited items but ignored by these methods. In this paper, we propose a novel Knowledge Based Prohibited item Detection system (named KBPD) to break through this limitation. To make full use of rich risk knowledge, the proposed method introduces the Risk-Domain Knowledge Graph (named RDKG), which is encoded by a path-based graph neural network method. Furthermore, to utilize information from both the RDKG and the Heterogeneous Risk Graph (named HRG), an interactive fusion framework is proposed and further improves the detection performance. We collect real-world datasets from the largest Chinese second-hand commodity trading platform, Xianyu. Both offline and online experimental results consistently demonstrate that KBPD outperforms the state-of-the-art baselines. The improvement over the second-best method is up to 22.67% in the AP metric.