OpenRaiser/Intern-Atlas huggingface dataset:
COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation
Authors: Yashal Shakti Kanungo, Gyanendra Das, Pooja A, Sumit Negi
Venue: KDD
Year: 2022

Online ads are essential to all businesses and ad headlines are one of their core creative component. Existing methods can generate headlines automatically and also optimize their click-through-rate (CTR) and quality. However, evolving ad formats and changing creative requirements make it difficult to generate optimized & customized headlines. We propose a novel method that uses prefix control tokens along with BART [16] fine-tuning. It yields the highest CTR and also allows users to control the length of generated headlines for use across different ad formats. The method is also flexible and can easily be adapted to other architectures, creative requirements and optimization criteria. Our experiments demonstrate a 25.82% increment in Rouge-L and a 5.82% increment in estimated CTR over previously published strong ad headline generation baseline.
Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Tasks
Authors: Sudipta Kar, Giuseppe Castellucci, Simone Filice, Shervin Malmasi, Oleg Rokhlenko
Venue: KDD
Year: 2022

Multi-Task Learning (MTL) is widely-accepted in Natural Language Processing as a standard technique for learning multiple related tasks in one model. Training an MTL model requires having the training data for all tasks available at the same time. As systems usually evolve over time, (e.g., to support new functionalities), adding a new task to an existing MTL model usually requires retraining the model from scratch on all the tasks and this can be time-consuming and computationally expensive. Moreover, in some scenarios, the data used to train the original training may be no longer available, for example, due to storage or privacy concerns.
ExMeshCNN: An Explainable Convolutional Neural Network Architecture for 3D Shape Analysis
Authors: Seonggyeom Kim, Dong-Kyu Chae
Venue: KDD
Year: 2022

Triangular meshes have been actively used in computer graphics to represent 3D shapes. However, due to their non-uniform and irregular nature, learning such data with a Deep Neural Network is not straightforward. Transforming mesh data to simpler structures (e.g., voxel grids, point clouds, or multi-view 2D images) leads to other issues including spatial information loss and scalability. Traditional descriptors for mesh data simply extract hand-crafted features, which might not be effective in various environments. Several deep architectures that directly consume mesh data have been proposed, but their input features are still heuristic and unable to fully capture both geodesic and geometric characteristics of a mesh. In addition, their model architectures are not designed to be capable of providing visual explanations of their decision making.
CoRGi: Content-Rich Graph Neural Networks with Attention
Authors: Jooyeon Kim, Angus Lamb, Simon Woodhead 0002, Simon Peyton Jones, Cheng Zhang 0005, Miltiadis Allamanis
Venue: KDD
Year: 2022

Graph representations of a target domain often project it to a set of entities (nodes) and their relations (edges). However, such projections often miss important and rich information. For example, in graph representations used in missing value imputation, items --- represented as nodes --- may contain rich textual information. However, when processing graphs with graph neural networks (GNN), such information is either ignored or summarized into a single vector representation used to initialize the GNN. Towards addressing this, we present CoRGi, a GNN that considers the rich data within nodes in the context of their neighbors. This is achieved by endowing CoRGi's message passing with a personalized attention mechanism over the content of each node. This way, CoRGi assigns user-item-specific attention scores with respect to the words that appear in an item's content. We evaluate CoRGi on two edge-value prediction tasks and show that CoRGi is better at making edge-value predictions over existing methods, especially on sparse regions of the graph.
The Sixth International Workshop on Automation in Machine Learning
Authors: Patrick Koch, Brett Wujek, Jun Liu 0003, Jun Huan, Tao Wang
Venue: KDD
Year: 2022

The Sixth International Workshop on Automation in Machine Learning aims to identify opportunities and challenges for automation in machine learning, to provide an opportunity for researchers to discuss best practices for automation in machine learning potentially leading to definition of standards, and to provide a forum for researchers to speak out and debate on different ideas in automation in machine learning. The workshop agenda includes four invited keynote speakers and four accepted paper presentations chosen from a peer review process. The workshop seeks to drive engaging and interactive exchange of thoughts and ideas on AutoML.
Self-Supervised Augmentation and Generation for Multi-lingual Text Advertisements at Bing
Authors: Xiaoyu Kou, Tianqi Zhao, Fan Zhang, Song Li, Qi Zhang
Venue: KDD
Year: 2022

Multi-lingual text advertisement generation is a critical task for international companies, such as Microsoft. Due to the lack of training data, scaling out text advertisements generation to low-resource languages is a grand challenge in the real industry setting. Although some methods transfer knowledge from rich-resource languages to low-resource languages through a pre-trained multi-lingual language model, they fail in balancing the transferability from the source language and the smooth expression in target languages. In this paper, we propose a unified Self-Supervised Augmentation and Generation (SAG) architecture to handle the multi-lingual text advertisements generation task in a real production scenario. To alleviate the problem of data scarcity, we employ multiple data augmentation strategies to synthesize training data in target languages. Moreover, a self-supervised adaptive filtering structure is developed to alleviate the impact of the noise in the augmented data. The new state-of-the-art results on a well-known benchmark verify the effectiveness and generalizability of our proposed framework, and deployment in Microsoft Bing demonstrates the superior performance of our method.
KDD Workshop on Machine Learning in Finance
Authors: Senthil Kumar, Leman Akoglu, Nitesh V. Chawla, Saurabh Nagrecha, Vidyut M. Naware, Tanveer A. Faruquie, Hays McCormick
Venue: KDD
Year: 2022

The finance industry is constantly faced with an ever evolving set of challenges including credit card fraud, identity theft, network intrusion, money laundering, human trafficking, and illegal sales of firearms. There is also the newly emerging threat of fake news in financial media that can lead to distortions in trading strategies and investment decisions. In addition, traditional problems such as customer analytics, forecasting, and recommendations take on a unique flavor when applied to financial data. A number of new ideas are emerging to tackle all these problems including semi-supervised learning methods, deep learning algorithms, network/graph based solutions as well as linguistic approaches. These methods must often be able to work in real-time and be able handle large volumes of data. The purpose of this workshop is to bring together researchers and practitioners to discuss both the problems faced by the financial industry and potential solutions. We plan to invite regular papers, positional papers and extended abstracts of work in progress. We will also encourage short papers from financial industry practitioners that introduce domain specific problems and challenges to academic researchers.
Variational Inference for Training Graph Neural Networks in Low-Data Regime through Joint Structure-Label Estimation
Authors: Danning Lao, Xinyu Yang 0002, Qitian Wu, Junchi Yan
Venue: KDD
Year: 2022

Graph Neural Networks (GNNs) are one of the prominent methods to solve semi-supervised learning on graphs. However, most of the existing GNN models often need sufficient observed data to allow for effective learning and generalization. In real-world scenarios where complete input graph structure and sufficient node labels might not be achieved easily, GNN models would encounter with severe performance degradation. To address this problem, we propose WSGNN, short for weakly-supervised graph neural network. WSGNN is a flexible probabilistic generative framework which harnesses variational inference approach to solve graph semi-supervised learning in a label-structure joint estimation manner. It collaboratively learns task-related new graph structure and node representations through a two-branch network, and targets a composite variational objective derived from underlying data generation distribution concerning the inter-dependence between scarce observed data and massive missing data. Especially, under weakly-supervised low-data regime where labeled nodes and observed edges are both very limited, extensive experimental results on node classification and link prediction over common benchmarks demonstrate the state-of-the-art performance of WSGNN over strong competitors. Concretely, when only 1 label per class and 1% edges are observed on Cora, WSGNN maintains a decent 52.00% classification accuracy, exceeding GCN by 75.6%.
The KDD 2022 Workshop on Causal Discovery (CD2022)
Authors: Thuc Duy Le, Lin Liu 0003, Emre Kiciman, Sofia Triantafyllou, Huan Liu 0001
Venue: KDD
Year: 2022

Causal relationships have been utilized in almost all disciplines, and the research into causal discovery has attracted a lot of attention in the last few years. Traditionally, causal relationships are identified by making use of interventions or randomized controlled experiments. However, conducting such experiments is often expensive or even impossible due to cost or ethical concerns. Therefore, there has been an increasing interest in discovering causal relationships based on observational data, and in the past few decades, significant contributions have been made to this field by computer scientists.
The DipEncoder: Enforcing Multimodality in Autoencoders
Authors: Collin Leiber, Lena G. M. Bauer, Michael Neumayr, Claudia Plant, Christian Böhm 0001
Venue: KDD
Year: 2022

Hartigan's Dip-test of unimodality gained increasing interest in unsupervised learning over the past few years. It is free from complex parameterization and does not require a distribution assumed a priori. A useful property is that the resulting Dip-values can be derived to find a projection axis that identifies multimodal structures in the data set. In this paper, we show how to apply the gradient not only with respect to the projection axis but also with respect to the data to improve the cluster structure. By tightly coupling the Dip-test with an autoencoder, we obtain an embedding that clearly separates all clusters in the data set. This method, called DipEncoder, is the basis of a novel deep clustering algorithm. Extensive experiments show that the DipEncoder is highly competitive to state-of-the-art methods.
Towards Learning Disentangled Representations for Time Series
Authors: Yuening Li, Zhengzhang Chen, Daochen Zha, Mengnan Du, Jingchao Ni, Denghui Zhang, Haifeng Chen, Xia Hu 0001
Venue: KDD
Year: 2022

Promising progress has been made toward learning efficient time series representations in recent years, but the learned representations often lack interpretability and do not encode semantic meanings by the complex interactions of many latent factors. Learning representations that disentangle these latent factors can bring semantic-rich representations of time series and further enhance interpretability. However, directly adopting the sequential models, such as Long Short-Term Memory Variational AutoEncoder (LSTM-VAE), would encounter a Kullback?Leibler (KL) vanishing problem: the LSTM decoder often generates sequential data without efficiently using latent representations, and the latent spaces sometimes could even be independent of the observation space. And traditional disentanglement methods may intensify the trend of KL vanishing along with the disentanglement process, because they tend to penalize the mutual information between the latent space and the observations. In this paper, we propose Disentangle Time-Series, a novel disentanglement enhancement framework for time series data. Our framework achieves multi-level disentanglement by covering both individual latent factors and group semantic segments. We propose augmenting the original VAE objective by decomposing the evidence lower-bound and extracting evidence linking factorial representations to disentanglement. Additionally, we introduce a mutual information maximization term between the observation space to the latent space to alleviate the KL vanishing problem while preserving the disentanglement property. Experimental results on five real-world IoT datasets demonstrate that the representations learned by DTS achieve superior performance in various tasks with better interpretability.
A Practical Introduction to Federated Learning
Authors: Yaliang Li, Bolin Ding, Jingren Zhou 0001
Venue: KDD
Year: 2022

As Internet users attach importance to their own privacy, and a number of laws and regulations go into effect in most countries, Internet products need to provide users with privacy protection. As one of the feasible solutions to provide such privacy protection, federated learning has rapidly gained popularity in both academia and industry in recent years. In this tutorial, we will start off with some real-world tasks to illustrate the topic of federated learning, and cover some basic concepts and important scenarios including cross-device and cross-silo settings. Along with it, we will give several demonstrations with popular federated learning frameworks. We will also show how to do the automatic hyperparameter tuning with federated learning to significantly save their efforts in practice. Then we dive into three parallel hot topics, Personalized Federated Learning, Federated Graph Learning, and Attack in Federated Learning. For each of them, we will motivate it with real-world applications, illustrate the state-of-the-art methods, and discuss their pros and cons using concrete examples. As the last part, we will point out some future research directions.
Design Domain Specific Neural Network via Symbolic Testing
Authors: Hui Li 0061, Xing Fu, Ruofan Wu, Jinyu Xu, Kai Xiao, Xiaofu Chang, Weiqiang Wang 0002, Shuai Chen, Leilei Shi, Tao Xiong, Yuan Qi 0001
Venue: KDD
Year: 2022

Deep sequence networks such as multi-head self-attention networks provide a promising way to extract effective representations from raw sequence data in an end-to-end fashion and have shown great success in various domains such as natural language processing, computer vision, $etc$. However, in domains such as financial risk management and anti-fraud where expert-derived features are heavily relied on, deep sequence models struggle to dominate the game.In this paper, we introduce a simple framework called symbolic testing to verify the learnability of certain expert-derived features over sequence data. A systematic investigation over simulated data reveals the fact that the self-attention architecture fails to learn some standard symbolic expressions like the count distinct operation. To overcome this deficiency, we propose a novel architecture named SHORING, which contains two components:event network andsequence network. Theevent network efficiently learns arbitrary high-orderevent-level conditional embeddings via a reparameterization trick while thesequence network integrates domain-specific aggregations into the sequence-level representation, thereby providing richer inductive biases compare to standard sequence architectures like self-attention. We conduct comprehensive experiments and ablation studies on synthetic datasets that mimic sequence data commonly seen in anti-fraud domain and three real-world datasets. The results show that SHORING learns commonly used symbolic features well, and experimentally outperforms the state-of-the-art methods by a significant margin over real-world online transaction datasets. The symbolic testing framework and SHORING have been applied in anti-fraud model development at Alipay and improved performance of models for real-time fraud-detection.
HierCDF: A Bayesian Network-based Hierarchical Cognitive Diagnosis Framework
Authors: Jiatong Li 0002, Fei Wang 0063, Qi Liu 0003, Mengxiao Zhu 0001, Wei Huang 0002, Zhenya Huang, Enhong Chen, Yu Su 0002, Shijin Wang 0001
Venue: KDD
Year: 2022

Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at quantifying students' cognitive level on knowledge attributes. Since there exists learning dependency among knowledge attributes, it is crucial for cognitive diagnosis models (CDMs) to incorporate attribute hierarchy when assessing students. The attribute hierarchy is only explored by a few CDMs such as Attribute Hierarchy Method, and there are still two significant limitations in these methods. First, the time complexity would be unbearable when the number of attributes is large. Second, the assumption used to model the attribute hierarchy is too strong so that it may lose some information of the hierarchy and is not flexible enough to fit all situations. To address these limitations, we propose a novel Bayesian network-based Hierarchical Cognitive Diagnosis Framework (HierCDF), which enables many traditional diagnostic models to flexibly integrate the attribute hierarchy for better diagnosis. Specifically, we first use an efficient Bayesian network to model the influence of attribute hierarchy on students' cognitive states. Then we design a CDM adaptor to bridge the gap between students' cognitive states and the input features of existing diagnostic models. Finally, we analyze the generality and complexity of HierCDF to show its effectiveness in modeling hierarchy information. The performance of HierCDF is experimentally proved on real-world large-scale datasets.
TaxoTrans: Taxonomy-Guided Entity Translation
Authors: Zhuliu Li, Yiming Wang, Xiao Yan, Weizhi Meng 0003, Yanen Li, Jaewon Yang
Venue: KDD
Year: 2022

Taxonomies describe the definitions of entities, entities' attributes and the relations among the entities, and thus play an important role in building a knowledge graph. In this paper, we tackle the task of taxonomy entity translation, which is to translate the names of taxonomy entities in a source language to a target language. The translations then can be utilized to build a knowledge graph in the target language. Despite its importance, taxonomy entity translation remains a hard problem for AI models due to two major challenges. One challenge is understanding the semantic context in very short entity names. Another challenge is having deep understanding for the domain where the knowledge graph is built.
Mining Spatio-Temporal Relations via Self-Paced Graph Contrastive Learning
Authors: Rongfan Li, Ting Zhong, Xinke Jiang, Goce Trajcevski, Jin Wu 0002, Fan Zhou 0002
Venue: KDD
Year: 2022

Modeling complex spatial and temporal dependencies are indispensable for location-bound time series learning. Existing methods, typically relying on graph neural networks (GNNs) and temporal learning modules based on recurrent neural networks, have achieved significant performance improvements. However, their representation capabilities and prediction results are limited when pre-defined graphs are unavailable. Unlike spatio-temporal GNNs focusing on designing complex architectures, we propose a novel adaptive graph construction strategy: Self-Paced Graph Contrastive Learning (SPGCL). It learns informative relations by maximizing the distinguishing margin between positive and negative neighbors and generates an optimal graph with a self-paced strategy. Specifically, the existing neighborhoods iteratively absorb more reliable nodes with the highest affinity scores as new neighbors to generate the next-round neighborhoods, and augmentations are applied to improve the transferability and robustness. As the adaptively self-paced graph approaches the optimized graph for prediction, the mutual information between nodes and the corresponding neighbors is maximized. Our work provides a new perspective of addressing spatio-temporal learning problems beyond information aggregation in Euclidean space and can be generalized to different tasks. Extensive experiments conducted on two typical spatio-temporal learning tasks (traffic forecasting and land displacement prediction) demonstrate the superior performance of SPGCL against the state-of-the-art.
Arbitrary Distribution Modeling with Censorship in Real-Time Bidding Advertising
Authors: Xu Li, Michelle Ma Zhang, Zhenya Wang, Youjun Tong
Venue: KDD
Year: 2022

The purpose of Inventory Pricing is to bid the right prices to online ad opportunities, which is crucial for a Demand-Side Platform (DSP) to win advertising auctions in Real-Time Bidding (RTB). In the planning stage, advertisers need the forecast of probabilistic models to make bidding decisions. However, most of the previous works made strong assumptions on the distribution form of the winning price, which reduced their accuracy and weakened their ability to make generalizations. Though some works recently tried to fit the distribution directly, their complex structure lacked efficiency on online inference, which is critical for advertising systems. In this paper, we devise a novel loss function, Neighborhood Likelihood Loss (NLL), collaborating with a proposed framework, Arbitrary Distribution Modeling (ADM), to predict the winning price distribution under censorship with no pre-assumption required. We conducted experiments on two real-world experimental datasets and one large-scale, non-simulated production dataset in our system. Experiments showed that ADM outperformed the baselines both on algorithm and business metrics. This method has been released for one year and led to good yield in our system. Without any pre-assumed specific distribution form, ADM showed significant advantages in effectiveness and efficiency, demonstrating its great capability in modeling sophisticated price landscapes.
A Logic Aware Neural Generation Method for Explainable Data-to-text
Authors: Xiexiong Lin, Huaisong Li, Tao Huang, Feng Wang, Linlin Chao, Fuzhen Zhuang, Taifeng Wang, Tianyi Zhang
Venue: KDD
Year: 2022

The most notable neural data-to-text approaches generate natural language from structural data relying on the surface form of the structural content, which ignores the underlying logical correlation between the input data and the target text. Moreover, identifying such logical associations and explaining them in natural language is desirable but not yet studied. In this paper, we introduce a practical data-to-text method for the logic-critical scenario, specifically for anti-money laundering applications. It involves detecting risks from input data and explaining any abnormal behaviors in natural language. The proposed method is a Logic Aware Neural Generation framework (LANG), which is a preliminary attempt to explore the integration of logic modeling and text generation. Concretely, we first convert expert rules to a logic graph. Then, the model utilizes meta path based encoder to exploit the expert knowledge. Besides, a retriever module with the encoded logic knowledge is used to bridge the gap between numeric input and target text. Finally, a rule-constrained loss is leveraged to improve the generation probability of tokens in rule recalled statements to ensure accuracy. We conduct extensive experiments on anti-money laundering data. Results show that the proposed method significantly outperforms baselines in both objective measures with relative 35% improvements in F1 score and subjective measures with 30% improvement in human preference.