OpenRaiser/Intern-Atlas huggingface dataset:
ACE-NODE: Attentive Co-Evolving Neural Ordinary Differential Equations
Authors: Sheo Yon Jhin, Minju Jo, Taeyong Kong, Jinsung Jeon, Noseong Park
Venue: KDD
Year: 2021

Neural ordinary differential equations (NODEs) presented a new paradigm to construct (continuous-time) neural networks. While showing several good characteristics in terms of the number of parameters and the flexibility in constructing neural networks, they also have a couple of well-known limitations: i) theoretically NODEs learn homeomorphic mapping functions only, and ii) sometimes NODEs show numerical instability in solving integral problems. To handle this, many enhancements have been proposed. To our knowledge, however, integrating attention into NODEs has been overlooked for a while. To this end, we present a novel method of attentive dual co-evolving NODE (ACE-NODE): one main NODE for a downstream machine learning task and the other for providing attention to the main NODE. Our ACE-NODE supports both pairwise and elementwise attention. In our experiments, our method outperforms existing NODE-based and non-NODE-based baselines in almost all cases by non-trivial margins.
Bootstrapping for Batch Active Sampling
Authors: Heinrich Jiang, Maya R. Gupta
Venue: KDD
Year: 2021

The goal of active learning is to select the best examples from an unlabeled pool of data to label to improve a model trained with the addition of these labeled examples. We discuss a real-world use case for batch active sampling that works at larger scales. The standard margin algorithm has repeatedly been shown difficult to beat in practice for the classic active sampling set-up, but for larger batches and candidate pools, we show that margin sampling may not provide enough diversity. We present a simple variant of margin sampling for the batch setting that scores candidate samples by their minimum margin to a set of bootstrapped margins, and explain how this proposal increases diversity in a supervised and efficient way, and why it differs from the usual ensemble methods for active sampling. Experiments on benchmark datasets show that the proposed min-margin sampling consistently works better than margin as the batch size grows, and better than the five other diversity-encouraging active sampling methods we tested. Two real-world case studies illustrate the practical value, and help highlight challenges of applying and deploying batch active sampling.
FleetRec: Large-Scale Recommendation Inference on Hybrid GPU-FPGA Clusters
Authors: Wenqi Jiang, Zhenhao He, Shuai Zhang, Kai Zeng, Liang Feng, Jiansong Zhang, Tongxuan Liu, Yong Li, Jingren Zhou, Ce Zhang, Gustavo Alonso
Venue: KDD
Year: 2021

We present FleetRec, a high-performance and scalable recommendation inference system within tight latency constraints. FleetRec takes advantage of heterogeneous hardware including GPUs and the latest FPGAs equipped with high-bandwidth memory. By disaggregating computation and memory to different types of hardware and bridging their connections by high-speed network, FleetRec gains the best of both worlds, and can naturally scale out by adding nodes to the cluster. Experiments on three production models up to 114 GB show that FleetRec outperforms optimized CPU baseline by more than one order of magnitude in terms of throughput while achieving significantly lower latency.
Multi-Armed Bandits and Reinforcement Learning: Advancing Decision Making in E-Commerce and Beyond
Authors: Daniel R. Jiang, Haipeng Luo, Chu Wang, Yingfei Wang
Venue: KDD
Year: 2021

The areas of reinforcement learning and multi-armed bandits have recently seen significant innovation, while many application domains, such as e-commerce, are full of problems and challenges to which vanilla RL or MAB methods cannot directly apply. This workshop aims at filling this communication gap by creating a platform for researchers and practitioners from both the method/theory side and application side of the community. Having this platform now instead of at a later time is beneficial to all sides of the community: practitioners and frontline scientists are able to avoid re-inventing existing techniques; theory-oriented researchers can find motivation in industry problems, working within more realistic settings, and making real-world impact. The 1st Multi-armed Bandits and Reinforcement Learning Workshop was a full day workshop co-located with the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2021) in Singapore.
Graph Representation Learning: Foundations, Methods, Applications and Systems
Authors: Wei Jin, Yao Ma, Yiqi Wang, Xiaorui Liu, Jiliang Tang, Yukuo Cen, Jiezhong Qiu, Jie Tang, Chuan Shi, Yanfang Ye, Jiawei Zhang, Philip S. Yu
Venue: KDD
Year: 2021

Graphs such as social networks and molecular graphs are ubiquitous data structures in the real world. Due to their prevalence, it is of great research importance to extract meaningful patterns from graph structured data so that downstream tasks can be facilitated. Instead of designing hand-engineered features, graph representation learning has emerged to learn representations that can encode the abundant information about the graph. It has achieved tremendous success in various tasks such as node classification, link prediction, and graph classification and has attracted increasing attention in recent years. In this tutorial, we systematically review the foundations, techniques, applications and advances in graph representation learning. We first introduce the foundations on graph theory and graph Fourier analysis. We then cover the key achievements of graph representation learning in recent years. Concretely, we discuss the six topics: 1) network embedding theories and systems; 2) foundations of graph neural networks (GNNs); 3) CogDL toolkit for GNNs; 4) scalable GNNs; 5) self-supervised learning in GNNs and 6) heterogeneous graphs and heterogeneous GNNs. Finally, we will introduce the applications of graph representation learning with a focus on recommender systems.
Learning to Walk across Time for Interpretable Temporal Knowledge Graph Completion
Authors: Jaehun Jung, Jinhong Jung, U Kang
Venue: KDD
Year: 2021

Static knowledge graphs (KGs), despite their wide usage in relational reasoning and downstream tasks, fall short of realistic modeling of knowledge and facts that are only temporarily valid. Compared to static knowledge graphs, temporal knowledge graphs (TKGs) inherently reflect the transient nature of real-world knowledge. Naturally, automatic TKG completion has drawn much research interests for a more realistic modeling of relational reasoning. However, most of the existing models for TKG completion extend static KG embeddings that do not fully exploit TKG structure, thus lacking in 1) accounting for temporally relevant events already residing in the local neighborhood of a query, and 2) path-based inference that facilitates multi-hop reasoning and better interpretability. In this paper, we propose T-GAP, a novel model for TKG completion that maximally utilizes both temporal information and graph structure in its encoder and decoder. T-GAP encodes query-specific substructure of TKG by focusing on the temporal displacement between each event and the query timestamp, and performs path-based inference by propagating attention through the graph. Our empirical experiments demonstrate that T-GAP not only achieves superior performance against state-of-the-art baselines, but also competently generalizes to queries with unseen timestamps. Through extensive qualitative analyses, we also show that T-GAP enjoys transparent interpretability, and follows human intuition in its reasoning process.
A Hyper-surface Arrangement Model of Ranking Distributions
Authors: Shizuo Kaji, Akira Horiguchi, Takuro Abe, Yohsuke Watanabe
Venue: KDD
Year: 2021

A distribution on the permutations over a fixed finite set is called a ranking distribution. Modelling ranking distributions is one of the major topics in preference learning as such distributions appear as the ranking data produced by many judges. In this paper, we propose a geometric model for ranking distributions. Our idea is to use hyper-surface arrangements in a metric space as the representation space, where each component cut out by hyper-surfaces corresponds to a total ordering, and its volume is proportional to the probability. In this setting, the union of components corresponds to a partial ordering and its probability is also estimated by the volume. Similarly, the probability of a partial ordering conditioned by another partial ordering is estimated by the ratio of volumes. We provide a simple iterative algorithm to fit our model to a given dataset. We show our model can represent the distribution of a real-world dataset faithfully and can be used for prediction and visualisation purposes.
Joint Graph Embedding and Alignment with Spectral Pivot
Authors: Paris A. Karakasis, Aritra Konar, Nicholas D. Sidiropoulos
Venue: KDD
Year: 2021

Graphs are powerful abstractions that naturally capture the wealth of relationships in our interconnected world. This paper proposes a new approach for graph alignment, a core problem in graph mining. Classical (e.g., spectral) methods use fixed embeddings for both graphs to perform the alignment. In contrast, the proposed approach fixes the embedding of the 'target' graph and jointly optimizes the embedding transformation and the alignment of the 'query' graph. An alternating optimization algorithm is proposed for computing high-quality approximate solutions and compared against the prevailing state-of-the-art graph aligning frameworks using benchmark real-world graphs. The results indicate that the proposed formulation can offer significant gains in terms of matching accuracy and robustness to noise relative to existing solutions for this hard but important problem.
Network Experimentation at Scale
Authors: Brian Karrer, Liang Shi, Monica Bhole, Matt Goldman, Tyrone Palmer, Charlie Gelman, Mikael Konutgan, Feng Sun
Venue: KDD
Year: 2021

We describe our network experimentation framework, deployed at Facebook, which accounts for interference between experimental units. We document this system, including the design and estimation procedures, and detail insights we have gained from the many experiments that have used this system at scale. In our estimation procedure, we introduce a cluster-based regression adjustment that substantially improves precision for estimating global treatment effects, as well as a procedure to test for interference. With our regression adjustment, we find that imbalanced clusters can better account for interference than balanced clusters without sacrificing accuracy. In addition, we show that logging exposure to a treatment can result in additional variance reduction. Interference is a widely acknowledged issue in online field experiments, yet there is less evidence from real-world experiments demonstrating interference in online settings. We fill this gap by describing two case studies that capture significant network effects and highlight the value of this experimentation framework.
Workshop on Data-Efficient Machine Learning (DeMaL)
Authors: Sumeet Katariya, Nikhil Rao, Chandan K. Reddy
Venue: KDD
Year: 2021

The recent increase in the size of neural networks has led to a proportional increase in the demands for high-quality human-annotated data. Labeling data is a costly and time-consuming endeavor, and the need for large data is often satiated through creative techniques such as data augmentation, transfer learning, self-supervised learning, active learning, to name a few. Many of these techniques are designed for specific data types such as images, text, and speech. The data in many data-mining applications however is multi-modal in nature, has implicit signals from user-interactions, and involves multiple agents. Given the uniqueness, importance, and growing interest in these problems, we feel that the ACM Conference on Knowledge Discovery and Data Mining (SIGKDD) 2021 is an appropriate venue for running a workshop on Data-efficient Machine Learning. In this proposal, we discuss our vision for this workshop.
Addressing Non-Representative Surveys using Multiple Instance Learning
Authors: Yaniv Katz, Oded Vainas
Venue: KDD
Year: 2021

In recent years, non representative survey sampling and non response bias constitute major obstacles in obtaining a reliable population quantity estimate from finite survey samples. As such, researchers have been focusing on identifying methods to resolve these biases. In this paper, we look at this well known problem from a fresh perspective, and formulate it as a learning problem. To meet this challenge, we suggest solving the learning problem using a multiple instance learning (MIL) paradigm. We devise two different MIL based neural network topologies, each based on a different implementation of an attention pooling layer. These models are trained to accurately infer the population quantity of interest even when facing a biased sample. To the best of our knowledge, this is the first time MIL has ever been suggested as a solution to this problem. In contrast to commonly used statistical methods, this approach can be accomplished without having to collect sensitive personal data of the respondents and without having to access population level statistics of the same sensitive data. To validate the effectiveness of our approaches, we test them on a real-world movie rating dataset which is used to mimic a biased survey by experimentally contaminating it with different kinds of survey bias. We show that our suggested topologies outperform other MIL architectures, and are able to partly counter the adverse effect of biased sampling on the estimation quality. We also demonstrate how these methods can be easily adapted to perform well even when part of the survey is based on a small number of respondents.
A Color-blind 3-Approximation for Chromatic Correlation Clustering and Improved Heuristics
Authors: Nicolas Klodt, Lars Seifert, Arthur Zahn, Katrin Casel, Davis Issac, Tobias Friedrich
Venue: KDD
Year: 2021

Chromatic Correlation Clustering (CCC) models clustering of objects with categorical pairwise relationships. The model can be viewed as clustering the vertices of a graph with edge-labels (colors). Bonchi et al. [KDD 2012] introduced it as a natural generalization of the well studied problem Correlation Clustering (CC), motivated by real-world applications from data-mining, social networks and bioinformatics. We give theoretical as well as practical contributions to the study of CCC. Our main theoretical contribution is an alternative analysis of the famous Pivot algorithm for CC. We show that, when simply run color-blind, Pivot is also a linear time 3-approximation for CCC. The previous best theoretical results for CCC were a 4-approximation with a high-degree polynomial runtime and a linear time 11-approximation, both by Anava et al. [WWW 2015]. While this theoretical result justifies Pivot as a baseline comparison for other heuristics, its blunt color-blindness performs poorly in practice. We develop a color-sensitive, practical heuristic we call Greedy Expansion that empirically outperforms all heuristics proposed for CCC so far, both on real-world and synthetic instances. Further, we propose a novel generalization of CCC allowing for multi-labelled edges. We argue that it is more suitable for many of the real-world applications and extend our results to this model.
2021 KDD Workshop on Understanding Public Perceptions for Applied Data Science: How Important is it to Engage Society in Technology Development?
Authors: ChanGhee Koh, SoYoung Kim, Nathaniel Tan
Venue: KDD
Year: 2021

The International Workshop on "Understanding Public Perceptions for Applied Data Science" (UPP4DS'21) presents interdisciplinary contributions that address AI trustworthiness from the perspective of the public as end-users. With healthcare as the context for discussion, the workshop will bring together leading experts in applied data science, healthcare, public policy, and social sciences to examine the role of public trust and public engagement in the development of healthcare AI applications. The workshop will also feature original research from the social sciences with significant implications in the adoption of healthcare technologies and invite open discussions on the role of society in the development of acceptable technologies.
Micro-climate Prediction - Multi Scale Encoder-decoder based Deep Learning Framework
Authors: Peeyush Kumar, Ranveer Chandra, Chetan Bansal, Shivkumar Kalyanaraman, Tanuja Ganu, Michael Grant
Venue: KDD
Year: 2021

This paper presents a deep learning approach for a versatile Micro-climate prediction framework (DeepMC). Micro climate predictions are of critical importance across various applications, such as Agriculture, Forestry, Energy, Search & Rescue, etc. To the best of our knowledge, there is no other single framework which can accurately predict various micro-climate entities using Internet of Things (IoT)data. We present a generic framework (DeepMC) which predicts various climatic parameters such as soil moisture, humidity, windspeed, radiation, temperature based on the requirement over a period of 12 hours - 120 hours with a varying resolution of 1 hour - 6hours, respectively. This framework proposes the following new ideas: 1) Localization of weather forecast to IoT sensors by fusing weather station forecasts with the decomposition of IoT data at multiple scales and 2) A multi-scale encoder and two levels of attention mechanisms which learns a latent representation of the interaction between various resolutions of the IoT sensor data and weather station forecasts. We present multiple real-world agricultural and energy scenarios, and report results with uncertainty estimates from the live deployment of DeepMC, which demonstrate that DeepMC outperforms various baseline methods and reports 90%+ accuracy with tight error bounds.
Dimensionwise Separable 2-D Graph Convolution for Unsupervised and Semi-Supervised Learning on Graphs
Authors: Qimai Li, Xiaotong Zhang, Han Liu, Quanyu Dai, Xiao-Ming Wu
Venue: KDD
Year: 2021

Graph convolutional neural networks (GCN) have been the model of choice for graph representation learning, which is mainly due to the effective design of graph convolution that computes the representation of a node by aggregating those of its neighbors. However, existing GCN variants commonly use 1-D graph convolution that solely operates on the object link graph without exploring informative relational information among object attributes. This significantly limits their modeling capability and may lead to inferior performance on noisy and sparse real-world networks. In this paper, we explore 2-D graph convolution to jointly model object links and attribute relations for graph representation learning. Specifically, we propose a computationally efficient dimensionwise separable 2-D graph convolution (DSGC) for filtering node features. Theoretically, we show that DSGC can reduce intra-class variance of node features on both the object dimension and the attribute dimension to learn more effective representations. Empirically, we demonstrate that by modeling attribute relations, DSGC achieves significant performance gain over state-of-the-art methods for node classification and clustering on a variety of real-world networks. The source code for reproducing the experimental results is available at https://github.com/liqimai/DSGC.
Architecture and Operation Adaptive Network for Online Recommendations
Authors: Lang Lang, Zhenlong Zhu, Xuanye Liu, Jianxin Zhao, Jixing Xu, Minghui Shan
Venue: KDD
Year: 2021

Learning feature interactions is crucial for model performance in online recommendations. Extensive studies are devoted to designing effective structures for learning interactive information in an explicit way and tangible progress has been made. However, the core interaction calculations of these models are artificially specified, such as inner product, outer product and self-attention, which results in high dependence on domain knowledge. Hence model effect is bounded by both restriction of human experience and the finiteness of candidate operations. In this paper, we propose a generalized interaction paradigm to lift the limitation, where operations adopted by existing models can be regarded as its special form. Based on this paradigm, we design a novel model to adaptively explore and optimize the operation itself according to data, named generalized interaction network(GIN). We proved that GIN is a generalized form of a wide range of state-of-the-art models, which means GIN can automatically search for the best operation among these models as well as a broader underlying architecture space. Finally, an architecture adaptation method is introduced to further boost the performance of GIN by discriminating important interactions. Thereby, architecture and operation adaptive network(AOANet) is presented. Experiment results on two large scale datasets show the superiority of our model. AOANet has been deployed to industrial production. In a 7-day A/B test, the click-through rate increased by 10.94%, which represents considerable business benefits.
The KDD 2021 Workshop on Causal Discovery (CD2021)
Authors: Thuc Duy Le, Jiuyong Li, Gregory F. Cooper, Sofia Triantafillou, Elias Bareinboim, Huan Liu, Negar Kiyavash
Venue: KDD
Year: 2021

As a basic and effective tool for explanation, prediction and decision making, causal relationships have been utilized in almost all disciplines. Traditionally, causal relationships are identified by making use of interventions or randomized controlled experiments. However, conducting such experiments is often expensive or even impossible due to cost or ethical concerns. Therefore, there has been an increasing interest in discovering causal relationships based on observational data, and in the past few decades, significant contributions have been made to this field by computer scientists. Inspired by such achievements and following the success of CD 2016 - CD 2020, CD 2021 continues to serve as a forum for researchers and practitioners in data mining and other disciplines to share their recent research in causal discovery in their respective fields and to explore the possibility of interdisciplinary collaborations in the study of causality. Based on the platform of KDD, this workshop is especially interested in attracting contributions that link data mining/machine learning research with causal discovery, and solutions to causal discovery in large-scale datasets.