OpenRaiser/Intern-Atlas huggingface dataset:
DyTed: Disentangled Representation Learning for Discrete-time Dynamic Graph
Authors: Kaike Zhang, Qi Cao 0005, Gaolin Fang, Bingbing Xu 0001, Hongjian Zou, Huawei Shen, Xueqi Cheng
Venue: KDD
Year: 2023

Unsupervised representation learning for dynamic graphs has attracted a lot of research attention in recent years. Compared with static graph, the dynamic graph is a comprehensive embodiment of both the intrinsic stable characteristics of nodes and the time-related dynamic preference. However, existing methods generally mix these two types of information into a single representation space, which may lead to poor explanation, less robustness, and a limited ability when applied to different downstream tasks. To solve the above problems, in this paper, we propose a novel disenTangled representation learning framework for discrete-time Dynamic graphs, namely DyTed. We specially design a temporal-clips contrastive learning task together with a structure contrastive learning to effectively identify the time-invariant and time-varying representations respectively. To further enhance the disentanglement of these two types of representation, we propose a disentanglement-aware discriminator under an adversarial learning framework from the perspective of information theory. Extensive experiments on Tencent and five commonly used public datasets demonstrate that DyTed, as a general framework that can be applied to existing methods, achieves state-of-the-art performance on various downstream tasks, as well as be more robust against noise.
Deep Transfer Learning for City-scale Cellular Traffic Generation through Urban Knowledge Graph
Authors: Shiyuan Zhang, Tong Li 0013, Shuodi Hui, Guangyu Li, Yanping Liang, Li Yu, Depeng Jin, Yong Li 0008
Venue: KDD
Year: 2023

The problem of cellular traffic generation in cities without historical traffic data is critical and urgently needs to be solved to assist 5G base station deployments in mobile networks. In this paper, we propose ADAPTIVE, a deep transfer learning framework for city-scale cellular traffic generation through the urban knowledge graph. ADAPTIVE leverages historical data from other cities that have deployed 5G networks to assist cities that are newly deploying 5G networks through deep transfer learning. Specifically, ADAPTIVE can align the representations of base stations in the target city and source city while considering the environmental factors of cities, spatial and environmental contextual relations between base stations, and traffic temporal patterns at base stations. We next design a feature-enhanced generative adversarial network, which is trained based on the historical traffic data and representations of base stations in the source city. By feeding the aligned target city's base station representations into the trained model, we can then obtain the generated traffic data for the target city. Extensive experiments on real-world cellular traffic datasets show that ADAPTIVE generally outperforms state-of-the-art baselines by more than 40% in terms of Jensen-Shannon divergence and root-mean-square error. Also, ADAPTIVE has strong robustness based on the results of various cross-city experiments. ADAPTIVE has been successfully deployed on the 'Jiutian' Artificial Intelligence Platform of China Mobile to support cellular traffic generation and assist in the construction and operation of mobile networks.
Towards a Generic Framework for Mechanism-guided Deep Learning for Manufacturing Applications
Authors: Hanbo Zhang, Jiangxin Li, Shen Liang, Peng Wang 0027, Themis Palpanas, Chen Wang 0018, Wei Wang 0009, Haoxuan Zhou, Jianwei Song, Wen Lu 0002
Venue: KDD
Year: 2023

Manufacturing data analytics tasks are traditionally undertaken with Mechanism Models (MMs), which are domain-specific mathematical equations modeling the underlying physical or chemical processes of the tasks. Recently, Deep Learning (DL) has been increasingly applied to manufacturing. MMs and DL have their individual pros and cons, motivating the development of Mechanism-guided Deep Learning Models (MDLMs) that combine the two. Existing MDLMs are often tailored to specific tasks or types of MMs, and can fail to effectively 1) utilize interconnections of multiple input examples, 2) adaptively self-correct prediction errors with error bounding, and 3) ensemble multiple MMs. In this work, we propose a generic, task-agnostic MDLM framework that can embed one or more MMs in deep networks, and address the 3 aforementioned issues. We present 2 diverse use cases where we experimentally demonstrate the effectiveness and efficiency of our models.
MixupExplainer: Generalizing Explanations for Graph Neural Networks with Data Augmentation
Authors: Jiaxing Zhang 0002, Dongsheng Luo, Hua Wei 0001
Venue: KDD
Year: 2023

Graph Neural Networks (GNNs) have received increasing attention due to their ability to learn from graph-structured data. However, their predictions are often not interpretable. Post-hoc instance-level explanation methods have been proposed to understand GNN predictions. These methods seek to discover substructures that explain the prediction behavior of a trained GNN. In this paper, we shed light on the existence of the distribution shifting issue in existing methods, which affects explanation quality, particularly in applications on real-life datasets with tight decision boundaries. To address this issue, we introduce a generalized Graph Information Bottleneck (GIB) form that includes a label-independent graph variable, which is equivalent to the vanilla GIB. Driven by the generalized GIB, we propose a graph mixup method, MixupExplainer, with a theoretical guarantee to resolve the distribution shifting issue. We conduct extensive experiments on both synthetic and real-world datasets to validate the effectiveness of our proposed mixup approach over existing approaches. We also provide a detailed analysis of how our proposed approach alleviates the distribution shifting issue.
A Personalized Automated Bidding Framework for Fairness-aware Online Advertising
Authors: Haoqi Zhang, Lvyin Niu, Zhenzhe Zheng 0001, Zhilin Zhang 0003, Shan Gu, Fan Wu 0006, Chuan Yu, Jian Xu 0015, Guihai Chen, Bo Zheng 0007
Venue: KDD
Year: 2023

Powered by machine learning techniques, online advertising platforms have launched various automated bidding strategy services to facilitate intelligent decision-making for advertisers. However, advertisers experience heterogeneous advertising environments, and thus the unified bidding strategies widely used in both academia and industry suffer from severe unfairness issues, resulting in significant ad performance disparity among advertisers. In this work, to resolve the unfairness issue and improve the overall system performance, we propose a personalized automated bidding framework, namely PerBid, shifting the classical automated bidding strategy with a unified agent to multiple context-aware agents corresponding to different advertiser clusters. Specifically, we first design an ad campaign profiling network to model dynamic advertising environments. By clustering the advertisers with similar profiles and generating context-aware automated bidding agents for each cluster, we can match advertisers with personalized automated bidding strategies. Experiments conducted on the real-world dataset and online A/B test on Alibaba display advertising platform demonstrate the effectiveness of PerBid in improving overall ad performance and guaranteeing fairness among heterogeneous advertisers.
Doubly Robust AUC Optimization against Noisy and Adversarial Samples
Authors: Chenkang Zhang, Wanli Shi, Lei Luo 0001, Bin Gu 0001
Venue: KDD
Year: 2023

Area under the ROC curve (AUC) is an important and widely used metric in machine learning especially for imbalanced datasets. In current practical learning problems, not only adversarial samples but also noisy samples seriously threaten the performance of learning models. Nowadays, there have been a lot of research works proposed to defend the adversarial samples and noisy samples separately. Unfortunately, to the best of our knowledge, none of them with AUC optimization can secure against the two kinds of harmful samples simultaneously. To fill this gap and also address the challenge, in this paper, we propose a novel doubly robust dAUC optimization (DRAUC) algorithm. Specifically, we first exploit the deep integration of self-paced learning and adversarial training under the framework of AUC optimization, and provide a statistical upper bound to the AUC adversarial risk. Inspired by the statistical upper bound, we propose our optimization objective followed by an efficient alternatively stochastic descent algorithm, which can effectively improve the performance of learning models by guarding against adversarial samples and noisy samples. Experimental results on several standard datasets demonstrate that our DRAUC algorithm has better noise robustness and adversarial robustness than the state-of-the-art algorithms.
Adaptive Disentangled Transformer for Sequential Recommendation
Authors: Yipeng Zhang 0003, Xin Wang 0019, Hong Chen 0011, Wenwu Zhu 0001
Venue: KDD
Year: 2023

Sequential recommendation aims at mining time-aware user interests through modeling sequential behaviors. Transformer, as an effective architecture designed to process sequential input data, has shown its superiority in capturing sequential relations for recommendation. Nevertheless, existing Transformer architectures lack explicit regularization for layer-wise disentanglement, which fails to take advantage of disentangled representation in recommendation and leads to suboptimal performance. In this paper, we study the problem of layer-wise disentanglement for Transformer architectures and propose the Adaptive Disentangled Transformer (ADT) framework, which is able to adaptively determine the optimal degree of disentanglement of attention heads within different layers. Concretely, we propose to encourage disentanglement by requiring the independence constraint via mutual information estimation over attention heads and employing auxiliary objectives to prevent the information from collapsing into useless noise. We further propose a progressive scheduler to adaptively adjust the weights controlling the degree of disentanglement via an evolutionary process. Extensive experiments on various real-world datasets demonstrate the effectiveness of our proposed ADT framework.
Domain-Specific Risk Minimization for Domain Generalization
Authors: Yifan Zhang 0004, Jindong Wang 0001, Jian Liang 0001, Zhang Zhang 0001, Baosheng Yu, Liang Wang 0001, Dacheng Tao, Xing Xie 0001
Venue: KDD
Year: 2023

Domain generalization (DG) approaches typically use the hypothesis learned on source domains for inference on the unseen target domain. However, such a hypothesis can be arbitrarily far from the optimal one for the target domain, induced by a gap termed ''adaptivity gap.'' Without exploiting the domain information from the unseen test samples, adaptivity gap estimation and minimization are intractable, which hinders us to robustify a model to any unknown distribution. In this paper, we first establish a generalization bound that explicitly considers the adaptivity gap. Our bound motivates two strategies to reduce the gap: the first one is ensembling multiple classifiers to enrich the hypothesis space, then we propose effective gap estimation methods for guiding the selection of a better hypothesis for the target. The other method is minimizing the gap directly by adapting model parameters using online target samples. We thus propose Domain-specific Risk Minimization (DRM). During training, DRM models the distributions of different source domains separately; for inference, DRM performs online model steering using the source hypothesis for each arriving target sample. Extensive experiments demonstrate the effectiveness of the proposed DRM for domain generalization. Code is available at: https://github.com/yfzhang114/AdaNPC.
Efficient Single-Source SimRank Query by Path Aggregation
Authors: Mingxi Zhang 0001, Yanghua Xiao, Wei Wang 0009
Venue: KDD
Year: 2023

Single-source SimRank query calculates the similarity between a query node and every node in a graph, which traverses the paths starting from the query node for similarity computation. However, the scale of the paths increases exponentially as path length increases, which decreases the computation efficiency. Sampling-based algorithms reduce computational cost by path sampling, but they need to sample sufficient paths to ensure the accuracy, and the performance might be affected by the large scale of paths. In this paper, we propose VecSim for efficient single-source SimRank query by path aggregation. VecSim first aggregates the paths starting from query node with common arrived nodes step by step to obtain the hitting probabilities, and then aggregates the paths starting from the arrived nodes reversely to obtain the first-meeting probabilities in a similar way, in which only several vectors are maintained. The extra-meeting probabilities are excluded from each step, and an efficient sampling-based algorithm is designed, which estimates the extra-meeting probabilities by sampling paths within a specified length. For further speeding up query processing, we propose a threshold-sieved algorithm, which prunes the entries with small values that contribute little to the final similarity scores by setting a threshold. Extensive experiments are done on four small and four large graphs, which demonstrate that VecSim outperforms the competitors in terms of time and space costs on a comparable accuracy. In particular, VecSim achieves an empirical error of 10-4 level in under 0.1 second over all of these graphs.
RelKD 2023: International Workshop on Resource-Efficient Learning for Knowledge Discovery
Authors: Chuxu Zhang, Dongkuan Xu, Mojan Javaheripi, Subhabrata Mukherjee, Lingfei Wu 0001, Yinglong Xia, Jundong Li, Meng Jiang 0001, Yanzhi Wang 0001
Venue: KDD
Year: 2023

Modern machine learning techniques, especially deep neural networks, have demonstrated excellent performance for various knowledge discovery and data mining applications. However, the development of many of these techniques still encounters resource constraint challenges in many scenarios, such as limited labeled data (data-level), small model size requirements in real-world computing platforms (model-level), and efficient mapping of the computations to heterogeneous target hardware (system-level). Addressing all of these metrics is critical for the effective and efficient usage of the developed models in a wide variety of real systems, such as large-scale social network analysis, large-scale recommendation systems, and real-time anomaly detection. Therefore, it is desirable to develop efficient learning techniques to tackle challenges of resource limitations from data, model/algorithm, or (and) system/hardware perspectives. The proposed international workshop on "Resource-Efficient Learning for Knowledge Discovery (RelKD 2023)" will provide a great venue for academic researchers and industrial practitioners to share challenges, solutions, and future opportunities of resource-efficient learning.
Efficient Approximation Algorithms for Spanning Centrality
Authors: Shiqi Zhang 0004, Renchi Yang, Jing Tang 0004, Xiaokui Xiao, Bo Tang 0016
Venue: KDD
Year: 2023

Given a graph \mathcalG , the spanning centrality (SC) of an edge e measures the importance of e for \mathcalG to be connected. In practice, SC has seen extensive applications in computational biology, electrical networks, and combinatorial optimization. However, it is highly challenging to compute the SC of all edges (AESC) on large graphs. Existing techniques fail to deal with such graphs, as they either suffer from expensive matrix operations or require sampling numerous long random walks. To circumvent these issues, this paper proposes TGT and its enhanced version TGT+, two algorithms for AESC computation that offers rigorous theoretical approximation guarantees. In particular, TGT remedies the deficiencies of previous solutions by conducting deterministic graph traversals with carefully-crafted truncated lengths. TGT+ further advances TGT in terms of both empirical efficiency and asymptotic performance while retaining result quality, based on the combination of TGT with random walks and several additional heuristic optimizations. We experimentally evaluate TGT+ against recent competitors for AESC using a variety of real datasets. The experimental outcomes authenticate that TGT+ outperforms state of the arts often by over one order of magnitude speedup without degrading the accuracy.
Hyperbolic Graph Topic Modeling Network with Continuously Updated Topic Tree
Authors: Delvin Ce Zhang, Rex Ying, Hady W. Lauw
Venue: KDD
Year: 2023

Connectivity across documents often exhibits a hierarchical network structure. Hyperbolic Graph Neural Networks (HGNNs) have shown promise in preserving network hierarchy. However, they do not model the notion of topics, thus document representations lack semantic interpretability. On the other hand, a corpus of documents usually has high variability in degrees of topic specificity. For example, some documents contain general content (e.g., sports), while others focus on specific themes (e.g., basketball and swimming). Topic models indeed model latent topics for semantic interpretability, but most assume a flat topic structure and ignore such semantic hierarchy. Given these two challenges, we propose a Hyperbolic Graph Topic Modeling Network to integrate both network hierarchy across linked documents and semantic hierarchy within texts into a unified HGNN framework. Specifically, we construct a two-layer document graph. Intra- and cross-layer encoding captures network hierarchy. We design a topic tree for text decoding to preserve semantic hierarchy and learn interpretable topics. Supervised and unsupervised experiments verify the effectiveness of our model.
Towards Disentangling Relevance and Bias in Unbiased Learning to Rank
Authors: Yunan Zhang 0001, Le Yan, Zhen Qin 0001, Honglei Zhuang, Jiaming Shen, Xuanhui Wang, Michael Bendersky, Marc Najork
Venue: KDD
Year: 2023

Unbiased learning to rank (ULTR) studies the problem of mitigating various biases from implicit user feedback data such as clicks, and has been receiving considerable attention recently. A popular ULTR approach for real-world applications uses a two-tower architecture, where click modeling is factorized into a relevance tower with regular input features, and a bias tower with bias-relevant inputs such as the position of a document. A successful factorization will allow the relevance tower to be exempt from biases. In this work, we identify a critical issue that existing ULTR methods ignored - the bias tower can be confounded with the relevance tower via the underlying true relevance. In particular, the positions were determined by the logging policy, i.e., the previous production model, which would possess relevance information. We give both theoretical analysis and empirical results to show the negative effects on relevance tower due to such a correlation. We then propose two methods to mitigate the negative confounding effects by better disentangling relevance and bias. Offline empirical results on both controlled public datasets and a large-scale industry dataset show the effectiveness of the proposed approaches. We conduct a live experiment on a popular web store for four weeks, and find a significant improvement in user clicks over the baseline, which ignores the negative confounding effect.
Local Boosting for Weakly-Supervised Learning
Authors: Rongzhi Zhang, Yue Yu 0001, Jiaming Shen, Xiquan Cui, Chao Zhang 0014
Venue: KDD
Year: 2023

Boosting is a commonly used technique to enhance the performance of a set of base models by combining them into a strong ensemble model. Though widely adopted, boosting is typically used in supervised learning where the data is labeled accurately. However, in weakly supervised learning, where most of the data is labeled through weak and noisy sources, it remains nontrivial to design effective boosting approaches. In this work, we show that the standard implementation of the convex combination of base learners can hardly work due to the presence of noisy labels. Instead, we propose LocalBoost, a novel framework for weakly-supervised boosting. LocalBoost iteratively boosts the ensemble model from two dimensions, i.e., intra-source and inter-source. The intra-source boosting introduces locality to the base learners and enables each base learner to focus on a particular feature regime by training new base learners on granularity-varying error regions. For the inter-source boosting, we leverage a conditional function to indicate the weak source where the sample is more likely to appear. To account for the weak labels, we further design an estimate-then-modify approach to compute the model weights. Experiments on seven datasets show that our method significantly outperforms vanilla boosting methods and other weakly-supervised methods.
GraphStorm an Easy-to-use and Scalable Graph Neural Network Framework: From Beginners to Heroes
Authors: Jian Zhang 0113, Da Zheng 0004, Xiang Song 0003, Theodore Vasiloudis, Israt Nisa, Jim Lu
Venue: KDD
Year: 2023

Applying Graph Neural Networks (GNNs) to real-world problems is challenging for machine learning (ML) practitioners due to two major obstacles. The first hurdle is the high barrier to learn programming GNNs from scratch. The second challenge lies in overcoming engineering difficulties when scaling GNN models for large graphs at an industry-level. To address these challenges, GraphStorm, an open-source framework, offers a solution by providing an easy-to-use user interface and an end-to-end GNN training/inference pipeline that seamlessly handles extremely large graphs in a distributed manner This tutorial aims to provide participants with a comprehensive understanding of GraphStorm, including its design principles, target users, and use cases, through presentations. The hands-on sections will enable attendees to walk through four practical GraphStorm use cases that can assist them in leveraging GNNs to address real-world business problems.
AdaProp: Learning Adaptive Propagation for Graph Neural Network based Knowledge Graph Reasoning
Authors: Yongqi Zhang, Zhanke Zhou, Quanming Yao, Xiaowen Chu 0001, Bo Han 0003
Venue: KDD
Year: 2023

Due to the popularity of Graph Neural Networks (GNNs), various GNN-based methods have been designed to reason on knowledge graphs (KGs). An important design component of GNN-based KG reasoning methods is called the propagation path, which contains a set of involved entities in each propagation step. Existing methods use hand-designed propagation paths, ignoring the correlation between the entities and the query relation. In addition, the number of involved entities will explosively grow at larger propagation steps. In this work, we are motivated to learn an adaptive propagation path in order to filter out irrelevant entities while preserving promising targets. First, we design an incremental sampling mechanism where the nearby targets and layer-wise connections can be preserved with linear complexity. Second, we design a learning-based sampling distribution to identify the semantically related entities. Extensive experiments show that our method is powerful, efficient and semantic-aware. The code is available at https://github.com/LARS-research/AdaProp.