OpenRaiser/Intern-Atlas huggingface dataset:
Overview of the 1st Workshop on City Brain Research
Authors: Guanjie Zheng, Porter Jenkins, Yanyan Xu, Dongyao Chen
Venue: KDD
Year: 2021

The 1st Workshop on City Brain Research examines the current challenges and recent breakthroughs related to intelligent urban transportation. The workshop will be organized in a novel form --- offering debates on three main components involved in the transportation policy development cycle: data collection, policy learning, and the effects on human behavior. The organizers intend to invite speakers and attendees from different backgrounds, ranging from computer science, transportation, to urban planning. The final outcomes include live discussions of the three consistent topics, a comprehensive annual report summarizing current practices and future directions, and a detailed tutorial on the workshop day.
Multi-view Denoising Graph Auto-Encoders on Heterogeneous Information Networks for Cold-start Recommendation
Authors: Jiawei Zheng, Qianli Ma, Hao Gu, Zhenjing Zheng
Venue: KDD
Year: 2021

Cold-start recommendation is a challenging problem due to the lack of user-item interactions. Recently, heterogeneous information network~(HIN)-based recommendation methods use rich auxiliary information to enhance users and items' connections, helping alleviate the cold-start problem. Despite progress, most existing methods model HINs under traditional supervised learning settings, ignoring the gaps between training and inference procedures in cold-start scenarios. In this paper, we regard cold-start recommendation as a missing data problem where some user-item interaction data are missing. Inspired by denoising auto-encoders that train a model to reconstruct the input from its corrupted version, we propose a novel model called Multi-view Denoising Graph Auto-Encoders~(MvDGAE) on HINS. Specifically, we first extract multifaceted meaningful semantics on HINs as multi-views for both users and items, effectively enhancing user/item relationships on different aspects. Then we conduct the training procedure by randomly dropping out some user-item interactions in the encoder while forcing the decoder to use these limited views to recover the full views, including the missing ones. In this way, the complementary representations for both users and items are more informative and robust to adjust to cold-start scenarios. Moreover, the decoder's reconstruction goals are multi-view user-user and item-item relationship graphs rather than the original input graphs, which make the features of similar users (or items) in the meta-paths closer together. Finally, we adopt a Bayesian task weight learner to balance multi-view graph reconstruction objectives automatically. Extensive experiments on both public benchmark datasets and a large-scale industry dataset WeChat Channel demonstrate that MvDGAE significantly outperforms the state-of-the-art recommendation models in various cold-start scenarios. The case studies also illustrate that MvDGAE has potentially good interpretability.
Multi-Objective Recommendations
Authors: Yong Zheng, David (Xuejun) Wang
Venue: KDD
Year: 2021

The development of recommender systems usually deal with single-objective optimizations, such as minimizing prediction errors or maximizing the ranking quality. There is an emerging demand in multi-objective recommendations in which the recommendation list can be generated by optimizing multiple objectives. For example, researchers may balance different evaluation metrics (e.g., accuracy, novelty, diversity) in their models, or consider different objectives in a multi-task recommender. This tutorial provides an overview of the multi-objective optimization and its applications in the area of recommender systems. More specifically, we summarize the multi-objective optimization methods, identify the circumstances in which a multi-objective recommender system could be useful, and point out the challenges in multi-objective recommendations.
Incorporating Prior Financial Domain Knowledge into Neural Networks for Implied Volatility Surface Prediction
Authors: Yu Zheng, Yongxin Yang, Bowei Chen
Venue: KDD
Year: 2021

In this paper we develop a novel neural network model for predicting implied volatility surface. Prior financial domain knowledge is taken into account. A new activation function that incorporates volatility smile is proposed, which is used for the hidden nodes that process the underlying asset price. In addition, financial conditions, such as the absence of arbitrage, the boundaries and the asymptotic slope, are embedded into the loss function. This is one of the very first studies which discuss a methodological framework that incorporates prior financial domain knowledge into neural network architecture design and model training. The proposed model outperforms the benchmarked models with the option data on the S&P 500 index over 20 years. More importantly, the domain knowledge is satisfied empirically, showing the model is consistent with the existing financial theories and conditions related to implied volatility surface.
Knowledge is Power: Hierarchical-Knowledge Embedded Meta-Learning for Visual Reasoning in Artistic Domains
Authors: Wenbo Zheng, Lan Yan, Chao Gou, Fei-Yue Wang
Venue: KDD
Year: 2021

This paper deals with the challenging problem of building visual reasoning models for answering questions related to artworks in artistic domains. The nature of abstract styles and cultural contexts within an artistic image makes the corresponding learning tasks extremely difficult. We propose a novel framework termed as Hierarchical-Knowledge Embedded Meta-Learning to address the critical issues of visual reasoning in artistic domains. In particular, we firstly present a deep relational model to capture and memorize the relations among different samples. Then, we provide the hierarchical-knowledge embedding that mines the implicit relationship between question-answer pairs for knowledge representation as the guidance of our meta-learner. This is a case of "knowledge is power" in the sense that the hierarchical knowledge representation is incorporated into our meta-learning based model. The final classification is derived from our model by learning to compare the features of samples. Experimental results show that our approach achieves significantly higher performance compared with other state-of-the-arts.
Accelerating Set Intersections over Graphs by Reducing-Merging
Authors: Weiguo Zheng, Yifan Yang, Chengzhi Piao
Venue: KDD
Year: 2021

Given two sets of vertices Sa and Sb of a graph, computing their common vertices, namely set intersection, is one primitive operation in many graph algorithms such as triangle counting, maximal clique enumeration, and subgraph matching. Thus, accelerating set intersections is beneficial to these algorithms. In the paper, we propose a novel reducing-merging framework for set intersections over graphs rather than intersecting the two sets directly. In the reducing phase, the vertices that cannot fall into the intersection are screened out by applying the range reduction. Based on the truncated subsets, the intersection can be easily obtained using the classic merging algorithm. To optimize the range codes that sketch the vertices, we formulate the problem of range code optimization and prove its NP-hardness. We develop efficient yet effective algorithms for two typical scenarios global intersection and local intersection. Moreover, we present a novel two-level merging algorithm to enhance the performance. The results of extensive experiments over real graphs show that our approach can achieve significant speedups compared to the merge-based algorithm.
Quantifying Assimilate-Contrast Effects in Online Rating Systems: Modeling, Analysis and Application
Authors: Mingze Zhong, Hong Xie, Qingsheng Zhu
Venue: KDD
Year: 2021

Online rating system serves as an indispensable building block for many web applications such as Amazon, TripAdvior and Yelp. It enables production quality estimation via aggregate ratings (a.k.a. wisdom of the crowd) as well as product recommendation via inferring user preference from ratings, etc. Previous studies showed that due to assimilate-contrast effects, historical ratings can significantly distort user's ratings, leading to low accuracy of product quality estimation and recommendation. To understand assimilate-contrast effects, an "accurate'' model is still missing as previous models do not capture important factors like rating recency, selection bias, etc. Furthermore, an analytical framework to characterize product estimation accuracy under assimilate-contrast effects is also missing. This paper aims to fill in this gap. We propose a mathematical model to quantify the aforementioned important factors on assimilate-contrast effects. Our model attains a good balance between model complexity and model accuracy, such that it is neat enough for us to develop an analytical framework to study assimilate-contrast effects. Based on our model, we derive sufficient conditions, under which the product estimation and collective opinion converges to the "ground-truth''. These conditions reveal important insights on how the aforementioned factors influence the convergence and guide the online rating system operator to design appropriate rating aggregation rules and rating displaying strategies. To demonstrate the versatility of our model, we apply to rating prediction tasks and product recommendation tasks. Experiment results on four public datasets show that our model can improve the rating prediction and and recommendation accuracy over previous models significantly.
Triplet Attention: Rethinking the Similarity in Transformers
Authors: Haoyi Zhou, Jianxin Li, Jieqi Peng, Shuai Zhang, Shanghang Zhang
Venue: KDD
Year: 2021

The Transformer model has benefited various real-world applications, where the self-attention mechanism with dot-products shows superior alignment ability on building long dependency. However, the pair-wisely attended self-attention limits further performance improvement on challenging tasks. To the extent of our knowledge, this is the first work to define the Triplet Attention (A3) for Transformer, which introduces triplet connections as the complementary dependency. Specifically, we define the triplet attention based on the scalar triplet product, which may be interchangeably used with the canonical one within the multi-head attention. It allows the self-attention mechanism to attend to diverse triplets and capture complex dependency. Then, we utilize the permuted formulation and kernel tricks to establish a linear approximation to A3. The proposed architecture could be smoothly integrated into the pre-training by modifying head configurations. Extensive experiments show that our methods achieve significant performance improvement on various tasks and two benchmarks.
Towards Fair Federated Learning
Authors: Zirui Zhou, Lingyang Chu, Changxin Liu, Lanjun Wang, Jian Pei, Yong Zhang
Venue: KDD
Year: 2021

Federated learning has become increasingly popular as it facilitates collaborative training of machine learning models among multiple clients while preserving their data privacy. In practice, one major challenge for federated learning is to achieve fairness in collaboration among the participating clients, because different clients' contributions to a model are usually far from equal due to various reasons. Besides, as machine learning models are deployed in more and more important applications, how to achieve model fairness, that is, to ensure that a trained model has no discrimination against sensitive attributes, has become another critical desiderata for federated learning. In this tutorial, we discuss formulations and methods such that collaborative fairness, model fairness, and privacy can be fully respected in federated learning. We review the existing efforts and the latest progress, and discuss a series of potential directions.
Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems
Authors: Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, Hongxia Yang
Venue: KDD
Year: 2021

Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning has become prevalent in industrial recommender systems. Standard approaches approximate maximum likelihood estimation (MLE) through sampling for better scalability and address the problem of DCG in a way similar to language modeling. However, live recommender systems face severe exposure bias and have a vocabulary several orders of magnitude larger than that of natural language, implying that MLE will preserve and even exacerbate the exposure bias in the long run in order to faithfully fit the observed samples. In this paper, we theoretically prove that a popular choice of contrastive loss is equivalent to reducing the exposure bias via inverse propensity weighting, which provides a new perspective for understanding the effectiveness of contrastive learning. Based on the theoretical discovery, we design CLRec, a contrastive learning method to improve DCG in terms of fairness, effectiveness and efficiency in recommender systems with extremely large candidate size. We further improve upon CLRec and propose Multi-CLRec, for accurate multi-intention aware bias reduction. Our methods have been successfully deployed in Taobao, where at least four-month online A/B tests and offline analyses demonstrate its substantial improvements, including a dramatic reduction in the Matthew effect.
PURE: Positive-Unlabeled Recommendation with Generative Adversarial Network
Authors: Yao Zhou, Jianpeng Xu, Jun Wu, Zeinab Taghavi Nasrabadi, Evren Körpeoglu, Kannan Achan, Jingrui He
Venue: KDD
Year: 2021

Recommender systems are powerful tools for information filtering with the ever-growing amount of online data. Despite its success and wide adoption in various web applications and personalized products, many existing recommender systems still suffer from multiple drawbacks such as large amount of unobserved feedback, poor model convergence, etc. These drawbacks of existing work are mainly due to the following two reasons: first, the widely used negative sampling strategy, which treats the unlabeled entries as negative samples, is invalid in real-world settings; second, all training samples are retrieved from the discrete observations, and the underlying true distribution of the users and items is not learned. In this paper, we address these issues by developing a novel framework named PURE, which trains an unbiased positive-unlabeled discriminator to distinguish the true relevant user-item pairs against the ones that are non-relevant, and a generator that learns the underlying user-item continuous distribution. For a comprehensive comparison, we considered 14 popular baselines from 5 different categories of recommendation approaches. Extensive experiments on two public real-world data sets demonstrate that PURE achieves the best performance in terms of 8 ranking based evaluation metrics.
DeepSpatial'21: 2nd International Workshop on Deep Learning for Spatiotemporal Data, Applications, and Systems
Authors: Xun Zhou, Liang Zhao, Zhe Jiang, Robert N. Stewart, Shashi Shekhar, Jieping Ye
Venue: KDD
Year: 2021

With the advancement of GPS and remote sensing technologies and the pervasiveness of smartphones and mobile devices, large amounts of spatiotemporal data are being collected from various domains. Knowledge discovery from spatiotemporal data is crucial in broad societal applications. Examples range from mapping flooded areas on satellite imagery for disaster response to monitoring crop health for food security, from estimating travel time between locations on Google Maps to forecasting hotspots of diseases like Covid-19 in public health. The recent success in deep learning technologies in computer vision and natural language processing provides unique opportunities for spatiotemporal data mining (e.g., automatically extracting spatial contextual features without manual feature engineering) but also faces unique challenges (e.g., spatial autocorrelation, heterogeneity, multiple scales, and resolutions, the existence of domain knowledge and constraints). This workshop provides a premium platform for researchers from both academia and industry to exchange ideas on opportunities, challenges, and cutting-edge techniques of deep learning for spatiotemporal data. We hope to inspire novel ideas and visions through the workshop and facilitate the development of this emerging research area.
3rd International Workshop on Deep Learning Practice for High-Dimensional Sparse Data with KDD 2021
Authors: Xiaoqiang Zhu, Kuang-chih Lee, Guorui Zhou, Biye Jiang, Zhe Wang, Ruiming Tang, Kan Ren, Qingyao Ai, Weinan Zhang
Venue: KDD
Year: 2021

Recently, we have witnessed that deep learning-based approaches has been widely applied to empower many internet-scale applications. However, the data in these internet-scale applications are high dimensional and extremely sparse, which makes it different from those applications with dense data processing, such as image classification and speech recognition, where deep learning-based approaches have been extensively studied. One of the main applications is the user-centric platform that consists of great deal of users, items and user generated tabular data which are quite high-dimensional. The characteristics of such data pose unique challenges to the adoption of deep learning in these applications, including modeling, training, and online serving, etc. More and more communities from both academia and industry have initiated the endeavors to solve these challenges. This workshop will provide a venue for both the research and engineering communities to discuss and formulate the challenges, utilize opportunities, and propose new ideas in the practice of deep learning on high-dimensional sparse data.
The Third International Workshop on Smart Data for Blockchain and Distributed Ledger (SDBD2021): Joint Workshop with SIGKDD 2021 Trust Day
Authors: Feida Zhu, Jian Pei
Venue: KDD
Year: 2021

Today's computing is characterized by an increasing degree of complexity, comprehensiveness and collaboration. The complexity can be observed by the wide application of gigantic models with a huge number of parameters and structures of an unprecedented level of sophistication. The comprehensiveness is best illustrated by the high heterogeneity of data both in terms of format and source. The collaboration, finally, becomes an obvious trend when computing systems grow more open and decentralized in which various entities interact to achieve collective intelligence with the presence of potentially malicious behavior. Trust, therefore, has become critical at multiple levels: At model level to assure its integrity, fairness and interpretability; At data level to safeguard data quality, compliance and privacy; At system level to govern resilience, performance and incentive. Moreover, the notion of trust has long been discussed in different domains in both academia and industry with different definition and understanding. The Third International Workshop on Smart Data for Blockchain and Distributed Ledger (SDBD'21) will be held as a joint workshop with the special-themed "Trust Day" of KDD 2021, which has therefore aimed to bring together researchers, practitioners and experts from various communities to exchange and explore ideas, frontiers, opportunities and challenges under the broad theme of "trust" in a highly interdisciplinary manner.
Adapting Pretrained Representations for Text Mining
Authors: Yu Meng 0001, Jiaxin Huang 0001, Yu Zhang 0044, Jiawei Han 0001
Venue: KDD
Year: 2022

Pretrained text representations, evolving from context-free word embeddings to contextualized language models, have brought text mining into a new era: By pretraining neural models on large-scale text corpora and then adapting them to task-specific data, generic linguistic features and knowledge can be effectively transferred to the target applications and remarkable performance has been achieved on many text mining tasks. Unfortunately, a formidable challenge exists in such a prominent pretrain-finetune paradigm: Large pretrained language models (PLMs) usually require a massive amount of training data for stable fine-tuning on downstream tasks, while human annotations in abundance can be costly to acquire.
International Workshop on Data-driven Science of Science
Authors: Yi Bu 0001, Meijun Liu, Yujia Zhai, Ying Ding 0001, Feng Xia 0001, Daniel E. Acuña, Yi Zhang 0095
Venue: KDD
Year: 2022

Citation data, along with other bibliographic datasets, have long been adopted by the knowledge and data discovery community as an important direction for presenting the validity and effectiveness of proposed algorithms and strategies. Many top computer scientists are also excellent researchers in the science of science. The purpose of this workshop is to bridge the two communities (i.e., the knowledge discovery community and the science of science community) together as the scholarly activities become salient web and social activities that start to generate a ripple effect on broader knowledge discovery communities. This workshop will showcase the current data-driven science of science research by highlighting several studies and constructing a community of researchers to explore questions critical to the future of data-driven science of science, especially a community of data-driven science of science in Data Science so as to facilitate collaboration and inspire innovation. Through discussion on emerging and critical topics in the science of science, this workshop aims to help generate effective solutions for addressing environmental, societal, and technological problems in the scientific community.
Why Data Scientists Prefer Glassbox Machine Learning: Algorithms, Differential Privacy, Editing and Bias Mitigation
Authors: Rich Caruana, Harsha Nori
Venue: KDD
Year: 2022

Recent research has shown that interpretable machine learning models can be just as accurate as blackbox learning methods on tabular datasets. In this tutorial we will walk you through leading open source tools for glassbox learning, and show how intelligible machine learning helps practitioners uncover flaws in their datasets, discover new science, and build models that are more fair and robust. We'll begin with an introduction to the science behind glassbox modeling, and walk through a series of case-studies that highlight the added value of interpretable methods in a variety of domains such as finance and healthcare without compromising accuracy. We'll also show how glassbox models can be used for state of the art differentially private learning, bias detection/mitigation, and how these models can be edited to remove undesirable effects with GAMChanger. We'll also discuss how to train interpretable models with deep neural nets.
21th International Workshop on Data Mining in Bioinformatics (BIOKDD 2022)
Authors: Da Yan 0001, Catia Pesquita, Carsten Görg, Jake Y. Chen, Mohammed J. Zaki
Venue: KDD
Year: 2022

The goal of the 21th International Workshop on Data Mining in Bioinformatics (BIOKDD 2022) is to encourage KDD researchers to solve the numerous problems and challenges in Bioinformatics using Data Mining technologies. Based on the organizers' expertise and communities, BIOKDD 2022 features 2 closely related themes "Biomedical Ontologies" and "Biological Data Visualization". These themes encourage the knowledge graph community and the data visualization community to work together to solve the many challenging problems in Bioinformatics with new perspectives. The key goal is to accelerate the convergence between Data Mining and Bioinformatics communities to expedite discoveries in basic biology, medicine and healthcare.