OpenRaiser/Intern-Atlas huggingface dataset:
When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting
Authors: Harshavardhan Kamarthi, Lingkai Kong, Alexander Rodríguez, Chao Zhang 0014, B. Aditya Prakash
Venue: KDD
Year: 2023

Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting, where the goal is to model and forecast multivariate time-series that have hierarchical relations. Previous works assume rigid consistency over the given hierarchies and do not adapt well to real-world data that show deviation from this assumption. Moreover, recent state-of-art neural probabilistic methods also impose hierarchical relations on point predictions and samples of the predictive distribution. This does not account for full forecast distributions being consistent with the hierarchy and leading to poorly calibrated forecasts. We close both these gaps and propose PROFHiT, a probabilistic hierarchical forecasting model that jointly models forecast distributions over the entire hierarchy. PROFHiT (1) uses a flexible probabilistic Bayesian approach and (2) introduces soft distributional consistency regularization that enables end-to-end learning of the entire forecast distribution leveraging information from the underlying hierarchy. This enables calibrated forecasts as well as adaptation to real-life data with varied hierarchical consistency. PROFHiT provides 41-88% better performance in accuracy and significantly better calibration over a wide range of dataset consistency. Furthermore, PROFHiT adapts to missing data and can provide reliable forecasts even if up to 10% of input time-series data is missing, whereas other methods' performance severely degrades by over 70%
Real Time Index and Search Across Large Quantities of GNN Experts for Low Latency Online Learning
Authors: Johan Kok Zhi Kang, Sien Yi Tan, Bingsheng He, Zhen Zhang 0023
Venue: KDD
Year: 2023

Online learning is a powerful technique that allows models to adjust to concept drift in dynamically changing graphs. This approach is crucial for large mobility-based companies like Grab, where batch-learning methods fail to keep up with the large amount of training data. Our work focuses on scaling graph neural network mixture of expert (MoE) models for real-time traffic speed prediction on road networks, while meeting high accuracy and low latency requirements. Conventional spatio-temporal and incremental MoE frameworks struggle with poor inference accuracy and linear time complexity when scaling experts, for the latter, leading to prohibitively high latency in model updates. To address this issue, we introduce the Indexed Router, a novel method that categorizes experts into a structured hierarchy called the indexed tree. This approach reduces the time to scale and search N number of experts from O(N) to O(log N), making it ideal for online learning under tight service level agreements. Our experiments show that these time savings do not compromise inference accuracy, and our Indexed Router outperforms state-of-the-art spatio-temporal and incremental MoE models in terms of traffic speed prediction accuracy on real-life GPS traces from Grab's database and publicly available records. In summary, the Indexed Router enables MoE models to scale across large numbers of experts with low latency, while accurately identifying the relevant experts for inference.
Getting an h-Index of 100 in 20 Years or Less!
Authors: Eamonn J. Keogh
Venue: KDD
Year: 2023

The title of this tutorial is clickbait! However, a high h-index (relative to the stage of your career), will make you advisor, chair, dean and chancellor happy, and it can help you get a job or get promoted. Moreover, I will show that h-indices are highly susceptible to the Matthew Effect, a high h-index may help you get funding and attract strong collaborators/students, which in turn will likely further increase your h-index! Thus, "kickstarting" your h-index early in your career can pay huge dividends.
Exploiting Relation-aware Attribute Representation Learning in Knowledge Graph Embedding for Numerical Reasoning
Authors: Gayeong Kim, Sookyung Kim, Ko Keun Kim, Suchan Park, Heesoo Jung, Hogun Park
Venue: KDD
Year: 2023

Numerical reasoning is an essential task for supporting machine learning applications, such as recommendation and information retrieval. The reasoning task aims to compare two items and infer new facts (e.g., is taller than) by leveraging existing relational information and numerical attributes (e.g., the height of an entity) in knowledge graphs. However, most existing methods rely on leveraging attribute encoders or additional loss functions to predict numerical relations. Therefore, the prediction performance is often not robust in cases when attributes are sparsely observed. In this paper, we propose a Relation-AAware attribute representation learning-based Knowledge Graph Embedding method for numerical reasoning tasks, which we call RAKGE. RAKGE incorporates a newly proposed attribute representation learning mechanism, which can leverage the association between relations and their corresponding numerical attributes. In addition, we introduce a robust self-supervised learning method to generate unseen positive and negative examples, thereby making our approach more reliable when numerical attributes are sparsely available. In the evaluation of three real-world datasets, our proposed model outperformed state-of-the-art methods, achieving an improvement of up to 65.1% in Hits@1 and up to 52.6% in MRR compared to the best competitor. Our implementation code is available at https://github.com/learndatalab/RAKGE.
Efficient Distributed Approximate k-Nearest Neighbor Graph Construction by Multiway Random Division Forest
Authors: Sang-Hong Kim, Ha-Myung Park
Venue: KDD
Year: 2023

k-nearest neighbor graphs, shortly k-NN graphs, are widely used in many data mining applications like recommendation, information retrieval, and similarity search. Approximate k-NN graph construction has been getting a lot of attention, and most researches focus on developing algorithms that operate efficiently and quickly on a single machine. A few pioneering studies propose distributed algorithms to increase the size of data that can be processed to billions. However, we notice that the distributed algorithms don't perform well enough due to the problems of graph fragmentation and massive data exchange. In this paper, we propose MRDF (Multiway Random Division Forest), a scalable distributed algorithm that constructs highly accurate k-NN graph from numerous high-dimensional vectors quickly. MRDF resolves the problems that the existing distributed algorithms suffer from, through coarse-grained partitioning based on tree path annotation. Experimental results on real-world datasets show that MRDF outperforms the state-of-the-art distributed algorithms with up to 7.6 times faster speed and up to 56%p better accuracy than the second best results.
LATTE: A Framework for Learning Item-Features to Make a Domain-Expert for Effective Conversational Recommendation
Authors: Taeho Kim 0003, Juwon Yu, Won-Yong Shin, Hyunyoung Lee, Ji-Hui Im, Sang-Wook Kim
Venue: KDD
Year: 2023

For high-quality conversational recommender systems (CRS), it is important to recommend the suitable items by capturing the items' features mentioned in the dialog and to explain the appropriate ones among the various features of the recommended item. We argue that the CRS model should be a domain-expert who is (1) knowledgeable about the relationships between items and their various features and (2) able to explain the recommended item with its features relevant to dialog context. To this end, we propose a novel framework, named as LATTE, to pre-train each core module in CRS (i.e., the recommendation and the conversation module) through abundant external data. For the recommendation module, we pre-train the recommendation module to comprehensively understand the relationships between items and their various features by leveraging both multi-reviews and a knowledge graph. For pre-training the conversation module, we create the synthetic dialogs, which contain responses providing the explanation relevant to the dialog context by using all the items' features and dialog templates. Through extensive experiments on two public CRS datasets, we demonstrate that LATTE exhibits (1) the effectiveness of each module in LATTE, (2) the superiority over 7 state-of-the art methods, and (3) the interpretations based on visualization.
Uncertainty Quantification in Deep Learning
Authors: Lingkai Kong, Harshavardhan Kamarthi, Peng Chen 0024, B. Aditya Prakash, Chao Zhang 0014
Venue: KDD
Year: 2023

Deep neural networks (DNNs) have achieved enormous success in a wide range of domains, such as computer vision, natural language processing and scientific areas. However, one key bottleneck of DNNs is that they are ignorant about the uncertainties in their predictions. They can produce wildly wrong predictions without realizing, and can even be confident about their mistakes. Such mistakes can cause misguided decisions-sometimes catastrophic in critical applications, ranging from self-driving cars to cyber security to automatic medical diagnosis. In this tutorial, we present recent advancements in uncertainty quantification for DNNs and their applications across various domains. We first provide an overview of the motivation behind uncertainty quantification, different sources of uncertainty, and evaluation metrics. Then, we delve into several representative uncertainty quantification methods for predictive models, including ensembles, Bayesian neural networks, conformal prediction, and others. We go on to discuss how uncertainty can be utilized for label-efficient learning, continual learning, robust decision-making, and experimental design. Furthermore, we showcase examples of uncertainty-aware DNNs in various domains, such as health, robotics, and scientific machine learning. Finally, we summarize open challenges and future directions in this area.
Deception by Omission: Using Adversarial Missingness to Poison Causal Structure Learning
Authors: Deniz Koyuncu, Alex Gittens, Bülent Yener, Moti Yung
Venue: KDD
Year: 2023

Causality-informed machine learning has been proposed as an avenue for achieving many of the goals of modern machine learning, from ensuring generalization under domain shifts to attaining fairness, robustness, and interpretability. A key component of causal machine learning is the inference of causal structures from observational data; in practice, this data may be incompletely observed. Prior work has demonstrated that adversarial perturbations of completely observed training data may be used to force the learning of inaccurate causal structural models (SCMs). However, when the data can be audited for correctness (e.g., it is cryptographically signed by its source), this adversarial mechanism is invalidated. This work introduces a novel attack methodology wherein the adversary deceptively omits a portion of the true training data to bias the learned causal structures in a desired manner (under strong signed sample input validation, this behavior seems to be the only strategy available to the adversary). Under this model, theoretically sound attack mechanisms are derived for the case of arbitrary SCMs, and a sample-efficient learning-based heuristic is given. Experimental validation of these approaches on real and synthetic data sets demonstrates the effectiveness of adversarial missingness attacks at deceiving popular causal structure learning algorithms.
Optimizing Traffic Control with Model-Based Learning: A Pessimistic Approach to Data-Efficient Policy Inference
Authors: Mayuresh Kunjir, Sanjay Chawla, Siddarth Chandrasekar, Devika Jay, Balaraman Ravindran
Venue: KDD
Year: 2023

Traffic signal control is an important problem in urban mobility with a significant potential for economic and environmental impact. While there is a growing interest in Reinforcement Learning (RL) for traffic signal control, the work so far has focussed on learning through simulations which could lead to inaccuracies due to simplifying assumptions. Instead, real experience data on traffic is available and could be exploited at minimal costs. Recent progress in offline or batch RL has enabled just that. Model-based offline RL methods, in particular, have been shown to generalize from the experience data much better than others.
The KDD'23 Workshop on Causal Discovery, Prediction and Decision (CDPD 2023)
Authors: Thuc Duy Le
Venue: KDD
Year: 2023

Causal relationships have been utilized in almost all disciplines, and the research into causal discovery has attracted a lot of attention in the last few years. Traditionally, causal relationships are identified by making use of interventions or randomized controlled experiments. However, conducting such experiments is often expensive or even impossible due to cost or ethical concerns. Therefore, there has been an increasing interest in discovering causal relationships based on observational data, and in the past few decades, significant contributions have been made to this field by computer scientists.
Causal Inference and Machine Learning in Practice: Use Cases for Product, Brand, Policy and Beyond
Authors: Jeong-Yoon Lee, Yifeng Wu, Keith Battocchi, Fabio Vera, Zhenyu Zhao, Totte Harinen, Jing Pan, Huigang Chen, Zeyu Zheng 0002, Chu Wang, Yingfei Wang, Xinwei Ma
Venue: KDD
Year: 2023

The increasing demand for data-driven decision-making has led to the rapid growth of machine learning applications in various industries. However, the ability to draw causal inferences from observational data remains a crucial challenge. In recent years, causal inference has emerged as a powerful tool for understanding the effects of interventions in complex systems. Combining causal inference with machine learning has the potential to provide a deeper understanding of the underlying mechanisms and to develop more effective solutions to real-world problems.
Mining of Real-world Hypergraphs: Patterns, Tools, and Generators
Authors: Geon Lee, Jaemin Yoo, Kijung Shin
Venue: KDD
Year: 2023

Group interactions are prevalent in various complex systems (e.g., collaborations of researchers and group discussions on online Q&A sites), and they are commonly modeled as hypergraphs. Hyperedges, which compose a hypergraph, are non-empty subsets of any number of nodes, and thus each hyperedge naturally represents a group interaction among entities. The higher-order nature of hypergraphs brings about unique structural properties that have not been considered in ordinary pairwise graphs.
Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networks
Authors: Gaotang Li, Marlena Duda, Xiang Zhang 0012, Danai Koutra, Yujun Yan
Venue: KDD
Year: 2023

Brain graphs, which model the structural and functional relationships between brain regions, are crucial in neuroscientific and clinical applications that can be formulated as graph classification tasks. However, dense brain graphs pose computational challenges such as large time and memory consumption and poor model interpretability. In this paper, we investigate effective designs in Graph Neural Networks (GNNs) to sparsify brain graphs by eliminating noisy edges. Many prior works select noisy edges based on explainability or task-irrelevant properties, but this does not guarantee performance improvement when using the sparsified graphs. Additionally, the selection of noisy edges is often tailored to each individual graph, making it challenging to sparsify multiple graphs collectively using the same approach.
Learning Balanced Tree Indexes for Large-Scale Vector Retrieval
Authors: Wuchao Li, Chao Feng 0008, Defu Lian, Yuxin Xie, Haifeng Liu 0004, Yong Ge, Enhong Chen
Venue: KDD
Year: 2023

Vector retrieval focuses on finding the k-nearest neighbors from a bunch of data points, and is widely used in a diverse set of areas such as information retrieval and recommender system. The current state-of-the-art methods represented by HNSW usually generate indexes with a big memory footprint, restricting the scale of data they can handle, except resorting to a hybrid index with external storage. The space-partitioning learned indexes, which only occupy a small memory, have made great breakthroughs in recent years. However, these methods rely on a large amount of labeled data for supervised learning, so model complexity affects the generalization.
Learning-Based Ad Auction Design with Externalities: The Framework and A Matching-Based Approach
Authors: Ningyuan Li 0001, Yunxuan Ma, Yang Zhao 0039, Zhijian Duan 0001, Yurong Chen 0002, Zhilin Zhang 0003, Jian Xu 0015, Bo Zheng 0007, Xiaotie Deng
Venue: KDD
Year: 2023

Learning-based ad auctions have increasingly been adopted in online advertising. However, existing approaches neglect externalities, such as the interaction between ads and organic items. In this paper, we propose a general framework, namely Score-Weighted VCG, for designing learning-based ad auctions that account for externalities. The framework decomposes the optimal auction design into two parts: designing a monotone score function and an allocation algorithm, which facilitates data-driven implementation. Theoretical results demonstrate that this framework produces the optimal incentive-compatible and individually rational ad auction under various externality-aware CTR models while being data-efficient and robust. Moreover, we present an approach to implement the proposed framework with a matching-based allocation algorithm. Experiment results on both real-world and synthetic data illustrate the effectiveness of the proposed approach.
What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders
Authors: Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen 0001, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, Weiqiang Wang 0002
Venue: KDD
Year: 2023

The last years have witnessed the emergence of a promising self-supervised learning strategy, referred to as masked autoencoding. However, there is a lack of theoretical understanding of how masking matters on graph autoencoders (GAEs). In this work, we present masked graph autoencoder (MaskGAE), a self-supervised learning framework for graph-structured data. Different from standard GAEs, MaskGAE adopts masked graph modeling (MGM) as a principled pretext task - masking a portion of edges and attempting to reconstruct the missing part with partially visible, unmasked graph structure. To understand whether MGM can help GAEs learn better representations, we provide both theoretical and empirical evidence to comprehensively justify the benefits of this pretext task. Theoretically, we establish close connections between GAEs and contrastive learning, showing that MGM significantly improves the self-supervised learning scheme of GAEs. Empirically, we conduct extensive experiments on a variety of graph benchmarks, demonstrating the superiority of MaskGAE over several state-of-the-arts on both link prediction and node classification tasks.
S2phere: Semi-Supervised Pre-training for Web Search over Heterogeneous Learning to Rank Data
Authors: Yuchen Li 0006, Haoyi Xiong, Linghe Kong, Qingzhong Wang, Shuaiqiang Wang, Guihai Chen, Dawei Yin 0001
Venue: KDD
Year: 2023

While Learning to Rank (LTR) models on top of transformers have been widely adopted to achieve decent performance, it is still challenging to train the model with sufficient data as only an extremely small number of query-webpage pairs could be annotated versus trillions of webpages available online and billions of web search queries everyday. In the meanwhile, industry research communities have released a number of open-source LTR datasets with well annotations but incorporating different designs of LTR features/labels (i.e., heterogeneous domains). In this work, inspired by the recent progress in pre-training transformers for performance advantages, we study the problem of pre-training LTR models using both labeled and unlabeled samples, especially we focus on the use of well-annotated samples in heterogeneous open-source LTR datasets to boost the performance of pre-training. Hereby, we propose S2phere-Semi-Supervised Pre-training with Heterogeneous LTR data strategies for LTR models using both unlabeled and labeled query-webpage pairs across heterogeneous LTR datasets. S2phere consists of a three-step approach: (1) Semi-supervised Feature Extraction Pre-training via Perturbed Contrastive Loss, (2) Cross-domain Ranker Pre-training over Heterogeneous LTR Datasets and (3) End-to-end LTR Fine-tuning via Modular Network Composition. Specifically, given an LTR model composed of a backbone (the feature extractor), a neck (the module to reason the orders) and a head (the predictor of ranking scores), S2phere uses unlabeled/labeled data from the search engine to pre-train the backbone in Step (1) via semi-supervised learning; then Step (2) incorporates multiple open-source heterogeneous LTR datasets to improve pre-training of the neck module as shared parameters of cross-domain learning; and finally, S2phere in Step (3) composes the backbone and neck with a randomly-initialized head into a whole LTR model and fine-tunes the model using search engine data with various learning strategies. Extensive experiments have been done with both offline experiments and online A/B Test on top of Baidu search engine. The comparisons against numbers of baseline algorithms confirmed the advantages of S2phere in producing high-performance LTR models for web-scale search.
Analysis of COVID-19 Offensive Tweets and Their Targets
Authors: Song Liao, Ebuka Okpala, Long Cheng 0005, Mingqi Li, Nishant Vishwamitra, Hongxin Hu, Feng Luo 0001, Matthew Costello
Venue: KDD
Year: 2023

During the global COVID-19 pandemic, people utilized social media platforms, especially Twitter, to spread and express opinions about the pandemic. Such discussions also drove the rise in COVID-related offensive speech. In this work, focusing on Twitter, we present a comprehensive analysis of COVID-related offensive tweets and their targets. We collected a COVID-19 dataset with over 747 million tweets for 30 months and fine-tuned a BERT classifier to detect offensive tweets. Our offensive tweets analysis shows that the ebb and flow of COVID-related offensive tweets potentially reflect events in the physical world. We then studied the targets of these offensive tweets. There was a large number of offensive tweets with abusive words, which could negatively affect the targeted groups or individuals. We also conducted a user network analysis, and found that offensive users interact more with other offensive users and that the pandemic had a lasting impact on some offensive users. Our study offers novel insights into the persistence and evolution of COVID-related offensive tweets during the pandemic