OpenRaiser/Intern-Atlas huggingface dataset:
RBG: Hierarchically Solving Large-Scale Routing Problems in Logistic Systems via Reinforcement Learning
Authors: Zefang Zong, Hansen Wang, Jingwei Wang 0002, Meng Zheng 0003, Yong Li 0008
Venue: KDD
Year: 2022

The large-scale vehicle routing problems (VRPs) are defined based on the classical VRPs with thousands of customers. It is an important optimization problem in modern logistic systems, since efficiently obtaining high-quality solutions can greatly reduce operation expenses as well as improve customer satisfaction. Most existing algorithms, including traditional non-learning heuristics and learning-based methods, only perform well on small-scale instances with usually no more than hundreds of customers. In this paper we present a novel Rewriting-by-Generating (RBG) framework which solves large-scale VRPs hierarchically. RBG consists of a rewriter agent that refines the customer division globally and an elementary generator to infer regional solutions locally. It is also flexible with multiple CVRP variant problems and could be continuously evolved with more up-to-date generator designs. We conduct extensive experiments on both synthetic and real-world data to demonstrate the effectiveness and efficiency of our proposed RBG framework. It outperforms HGS, one of the best heuristic method for CVRPs and also shortens the inference time. Online evaluation is also conducted on a deployed express platform in Guangdong, China, where RBG shows advantages to other alternative built-in algorithms.
22nd International Workshop on Data Mining in Bioinformatics (BIOKDD 2023)
Authors: Da Yan 0001, Ariful Azad, Jie Hou 0001, Jake Y. Chen, Mohammed J. Zaki
Venue: KDD
Year: 2023

The goal of the 22nd International Workshop on Data Mining in Bioinformatics (BIOKDD 2023) is to encourage KDD researchers to solve the numerous problems and challenges in Bioinformatics using Data Mining technologies. Based on the organizers' expertise and communities, BIOKDD 2023 features the theme ''Large-Scale Data-Driven Methods for Bioinformatics.'' This theme encourages the use of high-performance computing (HPC) to support the training of large machine learning models for problems in Bioinformatics and Computational Biology. The key goal is to accelerate the convergence between Data Mining and Bioinformatics communities to expedite discoveries in basic biology, medicine and healthcare.
Locality Sensitive Hashing for Optimizing Subgraph Query Processing in Parallel Computing Systems
Authors: Peng Peng 0001, Shengyi Ji, Zhen Tian, Hongbo Jiang 0001, Weiguo Zheng, Xuecang Zhang
Venue: KDD
Year: 2023

This paper explores parallel computing systems for efficient subgraph query processing in large graphs. We investigate how to take advantage of the inherent parallelism of parallel computing systems for both intraquery and interquery optimization during subgraph query processing. Rather than relying on widely-used hash-based methods, we utilize and extend locality sensitive hashing methods. For intraquery optimization, we use the structures of both the data graph and subgraph query to design a query-constraint locality sensitive hashing method named QCMH, which can be used to merge multiple tasks during a single subgraph query processing. For interquery optimization, we propose a query locality sensitive hashing method named QMH, which can be used to detect common subgraphs among different subgraph queries, thereby merging multiple subgraph queries. Our proposed methods can reduce the redundant computation among multiple tasks duringa single subgraph query processing or multiple queries. Extensive experimental studies on large real and synthetic graphs show that our proposed methods can improve query performance compared to state-of-the-art methods by 10% to 50%.
Graph Neural Processes for Spatio-Temporal Extrapolation
Authors: Junfeng Hu 0001, Yuxuan Liang 0002, Zhencheng Fan, Hongyang Chen 0001, Yu Zheng 0004, Roger Zimmermann
Venue: KDD
Year: 2023

We study the task of spatio-temporal extrapolation that generates data at target locations from surrounding contexts in a graph. This task is crucial as sensors that collect data are sparsely deployed, resulting in a lack of fine-grained information due to high deployment and maintenance costs. Existing methods either use learning-based models like Neural Networks or statistical approaches like Gaussian Processes for this task. However, the former lacks uncertainty estimates and the latter fails to capture complex spatial and temporal correlations effectively. To address these issues, we propose Spatio-Temporal Graph Neural Processes (STGNP), a neural latent variable model which commands these capabilities simultaneously. Specifically, we first learn deterministic spatio-temporal representations by stacking layers of causal convolutions and cross-set graph neural networks. Then, we learn latent variables for target locations through vertical latent state transitions along layers and obtain extrapolations. Importantly during the transitions, we propose Graph Bayesian Aggregation (GBA), a Bayesian graph aggregator that aggregates contexts considering uncertainties in context data and graph structure. Extensive experiments show that STGNP has desirable properties such as uncertainty estimates and strong learning capabilities, and achieves state-of-the-art results by a clear margin.
XAI for Predictive Maintenance
Authors: João Gama 0001, Slawomir Nowaczyk, Sepideh Pashami, Rita P. Ribeiro, Grzegorz J. Nalepa, Bruno Veloso
Venue: KDD
Year: 2023

The field of Explainable Predictive Maintenance (PM) is concerned with developing methods that can clarify how AI systems operate in the PM domain. One of the challenges of creating maintenance plans is integrating AI output with human decision-making pro- cesses and expertise. For AI to be helpful and trustworthy, fault predictions must be contextualized and easily comprehensible to humans. This involves providing tailored explanations to different actors depending on their roles and needs. For example, engineers can be connected to technical installation blueprints, while man- agers can evaluate system downtime costs, and lawyers can assess safety-threatening failures' potential liability. In many industries, black-box AI systems analyze sensor data to predict failures by detecting anomalies and deviations from typical behavior with impressive accuracy. However, PM is just one part of a broader context that aims to identify the most probable causes, develop a recovery plan, and estimate remaining useful life while providing alternative solutions. Achieving this requires complex interactions among various actors in industrial and decision-making processes. Our tutorial explores current trends, and promising research directions in Explainable AI (XAI) relevant to Explainable Predictive Maintenance (XPM), and future challenges and open issues on this topic. We will also present three case studies that highlight XPM's challenges in bus and train operations and steel factories.
The 3rd Workshop on Graph Learning Benchmarks (GLB 2023)
Authors: Jiaqi Ma 0001, Jiong Zhu, Yuxiao Dong, Danai Koutra, Jingrui He, Qiaozhu Mei, Anton Tsitsulin, Xingjian Zhang 0002, Marinka Zitnik
Venue: KDD
Year: 2023

Recent years have witnessed a surge of research interest in graph machine learning. However, the benchmark datasets available to the field are rather limited in both quantity and diversity, an issue particularly notable given the immense potential applications of graph learning. The lack of diverse benchmark datasets may have biased the development of graph machine learning techniques towards narrow directions. By crowdsourcing novel tasks and datasets, this workshop aims to increase the diversity of graph learning benchmarks, identify new demands of graph machine learning in general, and gain a better synergy of how concrete techniques perform on these benchmarks. Moreover, this workshop offers a platform for discussions of best practices in curating graph learning benchmarks and data-centric approaches for graph learning.
CF-GODE: Continuous-Time Causal Inference for Multi-Agent Dynamical Systems
Authors: Song Jiang 0002, Zijie Huang 0002, Xiao Luo 0001, Yizhou Sun
Venue: KDD
Year: 2023

Multi-agent dynamical systems refer to scenarios where multiple units (aka agents) interact with each other and evolve collectively over time. For instance, people's health conditions are mutually influenced. Receiving vaccinations not only strengthens the long-term health status of one unit but also provides protection for those in their immediate surroundings. To make informed decisions in multi-agent dynamical systems, such as determining the optimal vaccine distribution plan, it is essential for decision-makers to estimate the continuous-time counterfactual outcomes. However, existing studies of causal inference over time rely on the assumption that units are mutually independent, which is not valid for multi-agent dynamical systems. In this paper, we aim to bridge this gap and study how to estimate counterfactual outcomes in multi-agent dynamical systems. Causal inference in a multi-agent dynamical system has unique challenges: 1) Confounders are time-varying and are present in both individual unit covariates and those of other units; 2) Units are affected by not only their own but also others' treatments; 3) The treatments are naturally dynamic, such as receiving vaccines and boosters in a seasonal manner. To this end, we model a multi-agent dynamical system as a graph and propose a novel model called CF-GODE (C ounterFactual Graph Ordinary Differential Equations). CF-GODE is a causal model that estimates continuous-time counterfactual outcomes in the presence of inter-dependencies between units. To facilitate continuous-time estimation, we propose Treatment-Induced GraphODE, a novel ordinary differential equation based on graph neural networks (GNNs), which can incorporate dynamical treatments as additional inputs to predict potential outcomes over time. To remove confounding bias, we propose two domain adversarial learning based objectives that learn balanced continuous representation trajectories, which are not predictive of treatments and interference. We further provide theoretical justification to prove their effectiveness. Experiments on two semi-synthetic datasets confirm that CF-GODE outperforms baselines on counterfactual estimation. We also provide extensive analyses to understand how our model works.
Source-Free Domain Adaptation with Temporal Imputation for Time Series Data
Authors: Mohamed Ragab 0002, Emadeldeen Eldele, Min Wu 0008, Chuan-Sheng Foo, Xiaoli Li 0001, Zhenghua Chen
Venue: KDD
Year: 2023

Source-free domain adaptation (SFDA) aims to adapt a pretrained model from a labeled source domain to an unlabeled target domain without access to the source domain data, preserving source domain privacy. Despite its prevalence in visual applications, SFDA is largely unexplored in time series applications. The existing SFDA methods that are mainly designed for visual applications may fail to handle the temporal dynamics in time series, leading to impaired adaptation performance. To address this challenge, this paper presents a simple yet effective approach for source-free domain adaptation on time series data, namely MAsk and imPUte (MAPU). First, to capture temporal information of the source domain, our method performs random masking on the time series signals while leveraging a novel temporal imputer to recover the original signal from a masked version in the embedding space. Second, in the adaptation step, the imputer network is leveraged to guide the target model to produce target features that are temporally consistent with the source features. To this end, our MAPU can explicitly account for temporal dependency during the adaptation while avoiding the imputation in the noisy input space. Our method is the first to handle temporal consistency in SFDA for time series data and can be seamlessly equipped with other existing SFDA methods. Extensive experiments conducted on three real-world time series datasets demonstrate that our MAPU achieves significant performance gain over existing methods. Our code is available at: https://github.com/mohamedr002/MAPU_SFDA_TS.
Learning for Counterfactual Fairness from Observational Data
Authors: Jing Ma 0002, Ruocheng Guo, Aidong Zhang 0001, Jundong Li
Venue: KDD
Year: 2023

Fairness-aware machine learning has attracted a surge of attention in many domains, such as online advertising, personalized recommendation, and social media analysis in web applications. Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. Among many existing fairness notions, counterfactual fairness is a popular notion defined from a causal perspective. It measures the fairness of a predictor by comparing the prediction of each individual in the original world and that in the counterfactual worlds in which the value of the sensitive attribute is modified. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. However, in real-world scenarios, the underlying causal model is often unknown, and acquiring such human knowledge could be very difficult. In these scenarios, it is risky to directly trust the causal models obtained from information sources with unknown reliability and even causal discovery methods, as incorrect causal models can consequently bring biases to the predictor and lead to unfair predictions. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE. Specifically, under certain general assumptions, CLAIRE effectively mitigates the biases from the sensitive attribute with a representation learning framework based on counterfactual data augmentation and an invariant penalty. Experiments conducted on both synthetic and real-world datasets validate the superiority of CLAIRE in both counterfactual fairness and prediction performance.
How does the Memorization of Neural Networks Impact Adversarial Robust Models?
Authors: Han Xu 0002, Xiaorui Liu, Wentao Wang 0006, Zitao Liu 0001, Anil K. Jain 0001, Jiliang Tang
Venue: KDD
Year: 2023

Recent studies suggest that "memorization" is one necessary factor for overparameterized deep neural networks (DNNs) to achieve optimal performance. Specifically, the perfectly fitted DNNs can memorize the labels of many atypical samples, generalize their memorization to correctly classify test atypical samples and enjoy better test performance. While, DNNs which are optimized via adversarial training algorithms can also achieve perfect training performance by memorizing the labels of atypical samples, as well as the adversarially perturbed atypical samples. However, adversarially trained models always suffer from poor generalization, with both relatively low clean accuracy and robustness on the test set. In this work, we study the effect of memorization in adversarial trained DNNs and disclose two important findings: (a) Memorizing atypical samples is only effective to improve DNN's accuracy on clean atypical samples, but hardly improve their adversarial robustness and (b) Memorizing certain atypical samples will even hurt the DNN's performance on typical samples. Based on these two findings, we propose Benign Adversarial Training (BAT) which can facilitate adversarial training to avoid fitting "harmful" atypical samples and fit as more "benign" atypical samples as possible. In our experiments, we validate the effectiveness of BAT, and show that it can achieve better clean accuracy vs. robustness trade-off than baseline methods, in benchmark datasets for image classification.
TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter
Authors: Xinyang Zhang 0002, Yury Malkov, Omar Florez, Serim Park, Brian McWilliams, Jiawei Han 0001, Ahmed El-Kishky
Venue: KDD
Year: 2023

Pre-trained language models (PLMs) are fundamental for natural language processing applications. Most existing PLMs are not tailored to the noisy user-generated text on social media, and the pre-training does not factor in the valuable social engagement logs available in a social network. We present TwHIN-BERT, a multilingual language model productionized at Twitter, trained on in-domain data from the popular social network. TwHIN-BERT differs from prior pre-trained language models as it is trained with not only text-based self-supervision but also with a social objective based on the rich social engagements within a Twitter heterogeneous information network (TwHIN). Our model is trained on 7 billion tweets covering over 100 distinct languages, providing a valuable representation to model short, noisy, user-generated text. We evaluate our model on various multilingual social recommendation and semantic understanding tasks and demonstrate significant metric improvement over established pre-trained language models. We open-source TwHIN-BERT and our curated hashtag prediction and social engagement benchmark datasets to the research community.
Generalizing Graph ODE for Learning Complex System Dynamics across Environments
Authors: Zijie Huang 0002, Yizhou Sun, Wei Wang 0010
Venue: KDD
Year: 2023

Learning multi-agent system dynamics have been extensively studied for various real-world applications, such as molecular dynamics in biology, multi-body system prediction in physics, and particle dynamics in material science. Most of the existing models are built to learn single system dynamics, which learn the dynamics from observed historical data and predict the future trajectory. In practice, however, we might observe multiple systems that are generated across different environments, which differ in latent exogenous factors such as temperature and gravity. One simple solution is to learn multiple environment-specific models, but it fails to exploit the potential commonalities among the dynamics across environments and offers poor prediction results where per-environment data is sparse or limited. Here, we present GG-ODE (Generalized Graph Ordinary Differential Equations), a machine learning framework for learning continuous multi-agent system dynamics across environments. Our model learns system dynamics using neural ordinary differential equations (ODE) parameterized by Graph Neural Networks (GNNs) to capture the continuous interaction among agents. We achieve the model generalization by assuming the dynamics across different environments are governed by common physics laws that can be captured via learning a shared ODE function. The distinct latent exogenous factors learned for each environment are incorporated into the ODE function to account for their differences. To improve model performance, we additionally design two regularization losses to (1) enforce the orthogonality between the learned initial states and exogenous factors via mutual information minimization; and (2) reduce the temporal variance of learned exogenous factors within the same system via contrastive learning. Experiments over various physical simulations show that our model can accurately predict system dynamics, especially in the long range, and can generalize well to new systems with few observations.
Capacity Constrained Influence Maximization in Social Networks
Authors: Shiqi Zhang 0004, Yiqian Huang 0003, Jiachen Sun, Wenqing Lin, Xiaokui Xiao, Bo Tang 0016
Venue: KDD
Year: 2023

Influence maximization (IM) aims to identify a small number of influential individuals to maximize the information spread and finds applications in various fields. It was first introduced in the context of viral marketing, where a company pays a few influencers to promote the product. However, apart from the cost factor, the capacity of individuals to consume content poses challenges for implementing IM in real-world scenarios. For example, players on online gaming platforms can only interact with a limited number of friends. In addition, we observe that in these scenarios, (i) the initial adopters of promotion are likely to be the friends of influencers rather than the influencers themselves, and (ii) existing IM solutions produce sub-par results with high computational demands. Motivated by these observations, we propose a new IM variant called capacity constrained influence maximization (CIM), which aims to select a limited number of influential friends for each initial adopter such that the promotion can reach more users. To solve CIM effectively, we design two greedy algorithms, MG-Greedy and RR-Greedy, ensuring the 1/2-approximation ratio. To improve the efficiency, we devise the scalable implementation named RR-OPIM+ with (1/2-ε)-approximation and near-linear running time. We extensively evaluate the performance of 9 approaches on 6 real-world networks, and our solutions outperform all competitors in terms of result quality and running time. Additionally, we deploy RR-OPIM+ to online game scenarios, which improves the baseline considerably.
PlanRanker: Towards Personalized Ranking of Train Transfer Plans
Authors: Jia Xu 0005, Wanjie Tao, Zulong Chen, Jin Huang 0001, Huihui Liu, Hong Wen 0002, Shenghua Ni, Qun Dai, Yu Gu 0002
Venue: KDD
Year: 2023

Train transfer plan ranking has become the core business of online travel platforms (OTPs), due to the flourish development of high- speed rail technology and convenience of booking trains online. Currently, mainstream OTPs adopt rule-based or simple preference- based strategies to rank train transfer plans. However, the insuf- ficient emphasis on the costs of plans and the negligence of con- sidering reference transfer plans make these existing strategies less effective in solving the personalized ranking problem of train transfer plans. To this end, a novel personalized deep network (Plan- Ranker) is presented in this paper to better address the problem. In PlanRanker, a personalized learning component is first proposed to capture both of the query semantics and the target transfer plan- relevant personalized interests of a user over the user's behavior log data. Then, we present a cost learning component, where both of the price cost and the time cost of a target transfer plan are emphasized and learned. Finally, a reference transfer plan learning component is designed to enable the whole framework of PlanRanker to learn from reference transfer plans which are pieced together by plat- form users and thus reflect the wisdom of crowd. PlanRanker is now successfully deployed at Alibaba Fliggy, one of the largest OTPs in China, serving millions of users every day for train ticket reservation. Offline experiments on two production datasets and a country-scale online A/B test at Fliggy both demonstrate the superiority of the proposed PlanRanker over baselines.
Hyperbolic Graph Neural Networks: A Tutorial on Methods and Applications
Authors: Min Zhou 0006, Menglin Yang 0001, Bo Xiong 0001, Hui Xiong 0001, Irwin King
Venue: KDD
Year: 2023

Graph Neural Networks (GNNs) generalize conventional neural networks to graph-structured data and have received considerable attention owing to their impressive performance. In spite of the notable successes, the performance of Euclidean models is inherently bounded and limited by the representation ability of Euclidean geometry, especially when it comes to datasets with highly non-Euclidean latent anatomy. Recently, hyperbolic spaces have emerged as a promising alternative for processing graph data with tree-like structure or power-law distribution and a surge of works on either methods or novel applications have been seen. Unlike Euclidean space, which expands polynomially, hyperbolic space grows exponentially with its radius, making it more suitable for modeling complex real-world data. Hence, it gains natural advantages in abstracting tree-like graphs with a hierarchical organization or power-law distribution.