OpenRaiser/Intern-Atlas huggingface dataset:
Multi-Horizon Time Series Forecasting with Temporal Attention Learning
Authors: Chenyou Fan, Yuze Zhang, Yi Pan, Xiaoyue Li, Chi Zhang, Rong Yuan, Di Wu, Wensheng Wang, Jian Pei, Heng Huang
Venue: KDD
Year: 2019

We propose a novel data-driven approach for solving multi-horizon probabilistic forecasting tasks that predicts the full distribution of a time series on future horizons. We illustrate that temporal patterns hidden in historical information play an important role in accurate forecasting of long time series. Traditional methods rely on setting up temporal dependencies manually to explore related patterns in historical data, which is unrealistic in forecasting long-term series on real-world data. Instead, we propose to explicitly learn constructing hidden patterns' representations with deep neural networks and attending to different parts of the history for forecasting the future. In this paper, we propose an end-to-end deep-learning framework for multi-horizon time series forecasting, with temporal attention mechanisms to better capture latent patterns in historical data which are useful in predicting the future. Forecasts of multiple quantiles on multiple future horizons can be generated simultaneously based on the learned latent pattern features. We also propose a multimodal fusion mechanism which is used to combine features from different parts of the history to better represent the future. Experiment results demonstrate our approach achieves state-of-the-art performance on two large-scale forecasting datasets in different domains.
Training and Meta-Training Binary Neural Networks with Quantum Computing
Authors: Abdulah Fawaz, Paul Klein, Sebastien Piat, Simone Severini, Peter Mountney
Venue: KDD
Year: 2019

Quantum computers promise significant advantages over classical computers for a number of different applications. We show that the complete loss function landscape of a neural network can be represented as the quantum state output by a quantum computer. We demonstrate this explicitly for a binary neural network and, further, show how a quantum computer can train the network by manipulating this state using a well-known algorithm known as quantum amplitude amplification. We further show that with minor adaptation, this method can also represent the meta-loss landscape of a number of neural network architectures simultaneously. We search this meta-loss landscape with the same method to simultaneously train and design a binary neural network.
Hierarchical Multi-Task Word Embedding Learning for Synonym Prediction
Authors: Hongliang Fei, Shulong Tan, Ping Li
Venue: KDD
Year: 2019

Automatic synonym recognition is of great importance for entity-centric text mining and interpretation. Due to the high language use variability in real-life, manual construction of semantic resources to cover all synonyms is prohibitively expensive and may also result in limited coverage. Although there are public knowledge bases, they only have limited coverage for languages other than English. In this paper, we focus on medical domain and propose an automatic way to accelerate the process of medical synonymy resource development for Chinese, including both formal entities from healthcare professionals and noisy descriptions from end-users. Motivated by the success of distributed word representations, we design a multi-task model with hierarchical task relationship to learn more representative entity/term embeddings and apply them to synonym prediction. In our model, we extend the classical skip-gram word embedding model by introducing an auxiliary task "neighboring word semantic type prediction'' and hierarchically organize them based on the task complexity. Meanwhile, we incorporate existing medical term-term synonymous knowledge into our word embedding learning framework. We demonstrate that the embeddings trained from our proposed multi-task model yield significant improvement for entity semantic relatedness evaluation, neighboring word semantic type prediction and synonym prediction compared with baselines. Furthermore, we create a large medical text corpus in Chinese that includes annotations for entities, descriptions and synonymous pairs for future research in this direction.
Scaling Multi-Armed Bandit Algorithms
Authors: Edouard Fouché, Junpei Komiyama, Klemens Böhm
Venue: KDD
Year: 2019

The Multi-Armed Bandit (MAB) is a fundamental model capturing the dilemma between exploration and exploitation in sequential decision making. At every time step, the decision maker selects a set of arms and observes a reward from each of the chosen arms. In this paper, we present a variant of the problem, which we call the Scaling MAB (S-MAB): The goal of the decision maker is not only to maximize the cumulative rewards, i.e., choosing the arms with the highest expected reward, but also to decide how many arms to select so that, in expectation, the cost of selecting arms does not exceed the rewards. This problem is relevant to many real-world applications, e.g., online advertising, financial investments or data stream monitoring. We propose an extension of Thompson Sampling, which has strong theoretical guarantees and is reported to perform well in practice. Our extension dynamically controls the number of arms to draw. Furthermore, we combine the proposed method with ADWIN, a state-of-the-art change detector, to deal with non-static environments. We illustrate the benefits of our contribution via a real-world use case on predictive maintenance.
AtSNE: Efficient and Robust Visualization on GPU through Hierarchical Optimization
Authors: Cong Fu, Yonghui Zhang, Deng Cai, Xiang Ren
Venue: KDD
Year: 2019

Visualization of high-dimensional data is a fundamental yet challenging problem in data mining. These visualization techniques are commonly used to reveal the patterns in the high-dimensional data, such as clusters and the similarity among clusters. Recently, some successful visualization tools (e.g., BH-t-SNE and LargeVis) have been developed. However, there are two limitations with them : (1) they cannot capture the global data structure well. Thus, their visualization results are sensitive to initialization, which may cause confusions to the data analysis. (2) They cannot scale to large-scale datasets. They are not suitable to be implemented on the GPU platform because their complex algorithm logic, high memory cost, and random memory access mode will lead to low hardware utilization. To address the aforementioned problems, we propose a novel visualization approach named as Anchor-t-SNE (AtSNE), which provides efficient GPU-based visualization solution for large-scale and high-dimensional data. Specifically, we generate a number of anchor points from the original data and regard them as the skeleton of the layout, which holds the global structure information. We propose a hierarchical optimization approach to optimize the positions of the anchor points and ordinary data points in the layout simultaneously. Our approach presents much better and robust visual effects on 11 public datasets, and achieve 5 to 28 times speed-up on different datasets, compared with the current state-of-the-art methods. In particular, we deliver a high-quality 2-D layout for a 20 million and 96-dimension dataset within 5 hours, while the current methods fail to give results due to running out of the memory.
Conditional Random Field Enhanced Graph Convolutional Neural Networks
Authors: Hongchang Gao, Jian Pei, Heng Huang
Venue: KDD
Year: 2019

attention in recent years. Unlike the standard convolutional neural network, graph convolutional neural networks perform the convolutional operation on the graph data. Compared with the generic data, the graph data possess the similarity information between different nodes. Thus, it is important to preserve this kind of similarity information in the hidden layers of graph convolutional neural networks. However, existing works fail to do that. On the other hand, it is challenging to enforce the hidden layers to preserve the similarity relationship. To address this issue, we propose a novel CRF layer for graph convolutional neural networks to encourage similar nodes to have similar hidden features. In this way, the similarity information can be preserved explicitly. In addition, the proposed CRF layer is easy to compute and optimize. Therefore, it can be easily inserted into existing graph convolutional neural networks to improve their performance. At last, extensive experimental results have verified the effectiveness of our proposed CRF layer.
ProGAN: Network Embedding via Proximity Generative Adversarial Network
Authors: Hongchang Gao, Jian Pei, Heng Huang
Venue: KDD
Year: 2019

Network embedding has attracted increasing attention in recent few years, which is to learn a low-dimensional representation for each node of a network to benefit downstream tasks, such as node classification, link prediction, and network visualization. Essentially, the task of network embedding can be decoupled into discovering the proximity in the original space and preserving it in the low dimensional space. Only with the well-discovered proximity can we preserve it in the low-dimensional space. Thus, it is critical to discover the proximity between different nodes to learn good node representations. To address this problem, in this paper, we propose a novel proximity generative adversarial network (ProGAN) which can generate proximities. As a result, the generated proximity can help to discover the complicated underlying proximity to benefit network embedding. To generate proximities, we design a novel neural network architecture to fulfill it. In particular, the generation of proximities is instantiated to the generation of triplets of nodes, which encodes the similarity relationship between different nodes. In this way, the proposed ProGAN can generate proximities successfully to benefit network embedding. At last, extensive experimental results have verified the effectiveness of ProGAN.
LightNet: A Dual Spatiotemporal Encoder Network Model for Lightning Prediction
Authors: Yangli-ao Geng, Qingyong Li, Tianyang Lin, Lei Jiang, Liangtao Xu, Dong Zheng, Wen Yao, Weitao Lyu, Yijun Zhang
Venue: KDD
Year: 2019

Lightning as a natural phenomenon poses serious threats to human life, aviation and electrical infrastructures. Lightning prediction plays a vital role in lightning disaster reduction. Existing prediction methods, usually based on numerical weather models, rely on lightning parameterization schemes for forecasting. These methods, however, have two drawbacks. Firstly, simulations of the numerical weather models usually have deviations in space and time domains, which introduces irreparable biases to subsequent parameterization processes. Secondly, the lightning parameterization schemes are designed manually by experts in meteorology, which means these schemes can hardly benefit from abundant historical data. In this work, we propose a data-driven model based on neural networks, referred to as LightNet, for lightning prediction. Unlike the conventional prediction methods which are fully based on numerical weather models, LightNet introduces recent lightning observations in an attempt to calibrate the simulations and assist the prediction. LightNet first extracts spatiotemporal features of the simulations and observations via dual encoders. These features are then combined by a fusion module. Finally, the fused features are fed into a spatiotemporal decoder to make forecasts. We conduct experimental evaluations on a real-world North China lightning dataset, which shows that LightNet achieves a threefold improvement in equitable threat score for six-hour prediction compared with three established forecast methods.
Welfare Maximization in Online Two-sided Marketplaces
Authors: Sreenivas Gollapudi
Venue: KDD
Year: 2019

Recent years have witnessed the rise of many successful e-commerce marketplace platforms like AirBnB, Uber/Lyft, and Upwork, where a central platform mediates economic transactions between buyers and sellers. Some common features that distinguish such marketplaces from more traditional marketplaces are search and discovery of the service providers which could result in asymmetric matching of services; sharing of a service by multiple users such as ride-sharing; and handling different preferences such as patience, desired level of service expressed by participating agents. In this talk, I will summarize our work on different welfare maximizing strategies arising out of the aforementioned scenarios.
Seven Years of Data Science at Airbnb
Authors: Elena Grewal
Venue: KDD
Year: 2019

In this talk I will share my experience at the start of "Data Science" in 2012, joining one of the fastest growing early stage unicorns and then building data science over 7 years. What we think of when we think of data science has evolved rapidly during this time. At Airbnb we now use data across product and operations, and for decisions by humans and machines. I will share my predictions for the future and where I believe data science can make the biggest impact in the years to come.
Attribute-Driven Backbone Discovery
Authors: Sheng Guan, Hanchao Ma, Yinghui Wu
Venue: KDD
Year: 2019

Backbones refer to critical tree structures that span a set of nodes of interests in networks. This paper introduces a novel class of attributed backbones and detection algorithms in richly attributed networks. Unlike conventional models, attributed backbones capture dynamics in edge cost model: it specifies affinitive attributes for each edge, and the cost of each edge is dynamically determined by the selection of its associated affinitive attributes and the closeness of their values at its end nodes. The backbone discovery is to compute an attributed backbone that covers interested nodes with smallest connection cost dynamically determined by selected affinitive attributes. While this problem is hard to approximate, we develop feasible algorithms within practical reach for large attributed networks. (1) We show that this problem is fixed-parameter approximable parameterized by the number of affinitive attributes, by providing a Lagrangean-preserving 2-approximation. (2) When the attribute number is large and specifying closeness function is difficult, we provide a fast heuristic, which learns an edge-generative model, and applies the model to infer best backbones, without the need of specifying closeness functions. Using real-world networks, we verify the effectiveness and efficiency of our algorithms and show their applications in collaboration recommendation.
4 Perspectives in Human-Centered Machine Learning
Authors: Carlos Guestrin
Venue: KDD
Year: 2019

Machine learning (ML) has had a tremendous impact in across the world over the last decade. As we think about ML solving complex tasks, sometimes at super-human levels, it is easy to forget that there is no machine learning without humans in the loop. Humans define tasks and metrics, develop and program algorithms, collect and label data, debug and optimize systems, and are (usually) ultimately the users of the ML-based applications we are developing. In this talk, we will cover 4 human-centered perspectives in the ML development process, along with methods and systems, to empower humans to maximize the ultimate impact of their ML-based applications.
Deep Natural Language Processing for Search and Recommender Systems
Authors: Weiwei Guo, Huiji Gao, Jun Shi, Bo Long, Liang Zhang, Bee-Chung Chen, Deepak Agarwal
Venue: KDD
Year: 2019

Search and recommender systems share many fundamental components including language understanding, retrieval and ranking, and language generation. Building powerful search and recommender systems requires processing natural language effectively and efficiently. Recent rapid growth of deep learning technologies has presented both opportunities and challenges in this area. This tutorial offers an overview of deep learning based natural language processing (NLP) for search and recommender systems from an industry perspective. It first introduces deep learning based NLP technologies, including language understanding and language generation. Then it details how those technologies can be applied to common tasks in search and recommender systems, including query and document understanding, retrieval and ranking, and language generation. Applications in LinkedIn production systems are presented. The tutorial concludes with discussion of future trend.
Buying or Browsing?: Predicting Real-time Purchasing Intent using Attention-based Deep Network with Multiple Behavior
Authors: Long Guo, Lifeng Hua, Rongfei Jia, Binqiang Zhao, Xiaobo Wang, Bin Cui
Venue: KDD
Year: 2019

E-commerce platforms are becoming a primary place for people to find, compare and ultimately purchase products. One of the fundamental questions that arises in e-commerce is to predict user purchasing intent, which is an important part of user understanding and allows for providing better services for both sellers and customers. However, previous work cannot predict real-time user purchasing intent with a high accuracy, limited by the representation capability of traditional browse-interactive behavior adopted. In this paper, we propose a novel end-to-end deep network, named Deep Intent Prediction Network (DIPN), to predict real-time user purchasing intent. In particular, besides the traditional browse-interactive behavior, we collect a new type of user interactive behavior, called touch-interactive behavior, which can capture more fine-grained real-time user features. To combine these behavior effectively, we propose a hierarchical attention mechanism, where the bottom attention layer focuses on the inner parts of each behavior sequence while the top attention layer learns the inter-view relations between different behavior sequences. In addition, we propose to train DIPN with multi-task learning to better distinguish user behavior patterns. In the experiments conducted on a large-scale industrial dataset, DIPN significantly outperforms the baseline solutions. Notably, DIPN gains about 18.96% improvement on AUC than the state-of-the-art solution only using traditional browse-interactive behavior sequences. Moreover, DIPN has been deployed in the operational system of Taobao. Online A/B testing results with more than 12.9 millions of users reveal the potential of knowing users' real-time purchasing intent.
Applying Deep Learning to Airbnb Search
Authors: Malay Haldar, Mustafa Abdool, Prashant Ramanathan, Tao Xu, Shulin Yang, Huizhong Duan, Qing Zhang, Nick Barrow-Williams, Bradley C. Turnbull, Brendan M. Collins, Thomas Legrand
Venue: KDD
Year: 2019

The application to search ranking is one of the biggest machine learning success stories at Airbnb. Much of the initial gains were driven by a gradient boosted decision tree model. The gains, however, plateaued over time. This paper discusses the work done in applying neural networks in an attempt to break out of that plateau. We present our perspective not with the intention of pushing the frontier of new modeling techniques. Instead, ours is a story of the elements we found useful in applying neural networks to a real life product. Deep learning was steep learning for us. To other teams embarking on similar journeys, we hope an account of our struggles and triumphs will provide some useful pointers. Bon voyage!
A Deep Generative Approach to Search Extrapolation and Recommendation
Authors: Fred X. Han, Di Niu, Haolan Chen, Kunfeng Lai, Yancheng He, Yu Xu
Venue: KDD
Year: 2019

Related search query recommendation is a standard feature in many modern search engines. Interesting and relevant queries often increase the active time of users and improve the overall search experience. However, conventional approaches based on tag extraction, keywords matching or click graph link analysis suffer from the common problem of limited coverage and generalizability, which means the system could only make suggestions for a small portion of well-formed search queries. In this work, we propose a deep generative approach to construct a related search query for recommendation in a word-by-word fashion, given either an input query or the title of a document. We propose a novel two-stage learning framework that partitions the task into two simpler sub-problems, namely, relevant context words discovery and context-dependent query generation. We carefully design a Relevant Words Generator (RWG) based on recurrent neural networks and a Dual-Vocabulary Sequence-to-Sequence (DV-Seq2Seq) model to address these problems. We also propose automated strategies that have retrieved three large datasets with $500$K to $1$ million instances, from a search click graph constructed based on $8$ days of search histories in Tencent QQ Browser, for model training. By leveraging the dynamically discovered context words, our proposed framework outperforms other Seq2Seq generative baselines on a wide range of BLEU, ROUGE and Exact Match (EM) metrics.
GCN-MF: Disease-Gene Association Identification By Graph Convolutional Networks and Matrix Factorization
Authors: Peng Han, Peng Yang, Peilin Zhao, Shuo Shang, Yong Liu, Jiayu Zhou, Xin Gao, Panos Kalnis
Venue: KDD
Year: 2019

Discovering disease-gene association is a fundamental and critical biomedical task, which assists biologists and physicians to discover pathogenic mechanism of syndromes. With various clinical biomarkers measuring the similarities among genes and disease phenotypes, network-based semi-supervised learning (NSSL) has been commonly utilized by these studies to address this class-imbalanced large-scale data issue. However, most existing NSSL approaches are based on linear models and suffer from two major limitations: 1) They implicitly consider a local-structure representation for each candidate; 2) They are unable to capture nonlinear associations between diseases and genes. In this paper, we propose a new framework for disease-gene association task by combining Graph Convolutional Network (GCN) and matrix factorization, named GCN-MF. With the help of GCN, we could capture non-linear interactions and exploit measured similarities. Moreover, we define a margin control loss function to reduce the effect of sparsity. Empirical results demonstrate that the proposed deep learning algorithm outperforms all other state-of-the-art methods on most of metrics.
Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts
Authors: Junheng Hao, Muhao Chen, Wenchao Yu, Yizhou Sun, Wei Wang
Venue: KDD
Year: 2019

Many large-scale knowledge bases simultaneously represent two views of knowledge graphs (KGs): an ontology view for abstract and commonsense concepts, and an instance view for specific entities that are instantiated from ontological concepts. Existing KG embedding models, however, merely focus on representing one of the two views alone. In this paper, we propose a novel two-view KG embedding model, JOIE, with the goal to produce better knowledge embedding and enable new applications that rely on multi-view knowledge. JOIE employs both cross-view and intra-view modeling that learn on multiple facets of the knowledge base. The cross-view association model is learned to bridge the embeddings of ontological concepts and their corresponding instance-view entities. The intra-view models are trained to capture the structured knowledge of instance and ontology views in separate embedding spaces, with a hierarchy-aware encoding technique enabled for ontologies with hierarchies. We explore multiple representation techniques for the two model components and investigate with nine variants of JOIE. Our model is trained on large-scale knowledge bases that consist of massive instances and their corresponding ontological concepts connected via a (small) set of cross-view links. Experimental results on public datasets show that the best variant of JOIE significantly outperforms previous models on instance-view triple prediction task as well as ontology population on ontology-view KG. In addition, our model successfully extends the use of KG embeddings to entity typing with promising performance.