OpenRaiser/Intern-Atlas huggingface dataset:
Augmenting Recurrent Graph Neural Networks with a Cache
Authors: Guixiang Ma, Vy A. Vo, Theodore L. Willke, Nesreen K. Ahmed
Venue: KDD
Year: 2023

While graph neural networks (GNNs) provide a powerful way to learn structured representations, it remains challenging to learn long-range dependencies in graphs. Recurrent GNNs only partly address this problem. In this paper, we propose a general approach for augmenting recurrent GNNs with a cache memory to improve their expressivity, especially for modeling long-range dependencies. Specifically, we first introduce a method of augmenting recurrent GNNs with a cache of previous hidden states. Then we further propose a general Cache-GNN framework by adding additional modules, including attention mechanism and positional/structural encoders, to improve the expressivity. We show that the Cache-GNNs outperforms other models on synthetic datasets as well as tasks on real-world datasets that require long-range information.
Multi-Label Learning to Rank through Multi-Objective Optimization
Authors: Debabrata Mahapatra, Chaosheng Dong, Yetian Chen, Michinari Momma
Venue: KDD
Year: 2023

Learning to Rank (LTR) technique is ubiquitous in Information Retrieval systems, especially in search ranking applications. The relevance labels used to train ranking models are often noisy measurements of human behavior, such as product ratings in product searches. This results in non-unique ground truth rankings and ambiguity. To address this, Multi-Label LTR (MLLTR) is used to train models using multiple relevance criteria, capturing conflicting but important goals, such as product quality and purchase likelihood for improved revenue in product searches. This research leverages Multi-Objective Optimization (MOO) in MLLTR and employs modern MOO algorithms to solve the problem. A general framework is proposed to combine label information to characterize trade-offs among goals, and allows for the use of gradient-based MOO algorithms. We test the proposed framework on four publicly available LTR datasets and one E-commerce dataset to show its efficacy.
Querywise Fair Learning to Rank through Multi-Objective Optimization
Authors: Debabrata Mahapatra, Chaosheng Dong, Michinari Momma
Venue: KDD
Year: 2023

In Learning-to-Rank (LTR) problems, the task of delivering relevant search results and allocating fair exposure to items of a protected group can conflict. Previous works in Fair LTR have attempted to resolve this by combining the objectives of relevant ranking and fair ranking into a single linear combination, but this approach is limited by the nonconvexity of the objective functions and can result in suboptimal relevance in ranking outputs. To address this, we propose a solution using Multi-Objective Optimization (MOO) algorithms. We extend these algorithms to querywise MOO to reduce the exposure disparity, not only on average but also at the query level. Interestingly, for moderate fairness requirements, it improves the relevance of ranking instead of deteriorating. We attribute this improvement to the benefits of multi-task learning and study the effect of fair ranking on the relevant ranking task. Moreover, we significantly improve the computational efficiency compared to previous methods by using the Gumbel max trick to sample the Plackett-Luce distribution. We evaluate our proposed methods on three real-world datasets and show their improvement in relevance ranking over state-of-the-art solutions.
The 6th Workshop on e-eommerce and NLP (ECNLP 6)
Authors: Shervin Malmasi, Besnik Fetahu, Eugene Agichtein, Oleg Rokhlenko, Ido Guy, Nicola Ueffing, Surya Kallumadi
Venue: KDD
Year: 2023

Natural Language Processing (NLP) technology plays a key role in e-commerce today, where this technology can be used for a range of tasks, such as improving search results, providing recommendations, and powering virtual assistants. The ECNLP workshop series focuses on NLP and Machine Learning methods for e-commerce, with a focus on applied and fundamental machine learning and NLP methods that can be leveraged in applied settings. The workshop aims to being together researchers from both industry and academia, with the goal of fostering greater knowledge sharing and collaboration between researchers and practitioners in this field.
Fast Text Generation with Text-Editing Models
Authors: Eric Malmi, Yue Dong 0002, Jonathan Mallinson, Aleksandr Chuklin, Jakub Adámek, Daniil Mirylenka, Felix Stahlberg, Sebastian Krause, Shankar Kumar, Aliaksei Severyn
Venue: KDD
Year: 2023

Text-editing models have recently become a prominent alternative to seq2seq models for monolingual text-generation tasks such as grammatical error correction, simplification, and style transfer. These tasks share a common trait -- they exhibit a large amount of textual overlap between the source and target texts. Text-editing models take advantage of this observation and learn to generate the output by predicting edit operations applied to the source sequence. In contrast, seq2seq models generate outputs word-by-word from scratch thus making them slow at inference time. Text-editing models provide several benefits over seq2seq models including faster inference speed, higher sample efficiency, and better control and explainability of the outputs. This tutorial provides a comprehensive overview of text-editing models and discusses how they can be used to mitigate hallucination and bias, both pressing challenges in the field of text generation. Finally, we discuss how to optimize latency of large language models via distillation to text-editing models and other means.
Detecting Vulnerable Nodes in Urban Infrastructure Interdependent Network
Authors: Jinzhu Mao, Liu Cao, Chen Gao 0001, Huandong Wang, Hangyu Fan, Depeng Jin, Yong Li 0008
Venue: KDD
Year: 2023

Understanding and characterizing the vulnerability of urban infrastructures, which refers to the engineering facilities essential for the regular running of cities and that exist naturally in the form of networks, is of great value to us. Potential applications include protecting fragile facilities and designing robust topologies, etc. Due to the strong correlation between different topological characteristics and infrastructure vulnerability and their complicated evolution mechanisms, some heuristic and machine assisted analysis fall short in addressing such a scenario. In this paper, we model the interdependent network as a heterogeneous graph and propose a system based on graph neural network with reinforcement learning, which can be trained on real-world data, to characterize the vulnerability of the city system accurately. The presented system leverages deep learning techniques to understand and analyze the heterogeneous graph, which enables us to capture the risk of cascade failure and discover vulnerable infrastructures of cities. Extensive experiments with various requests demonstrate not only the expressive power of our system but also transferring ability and necessity of the specific components. All source codes and models including those that can reproduce all figures analyzed in this work are publicly available at this link: https://github.com/tsinghua-fib-lab/KDD2023-ID546-UrbanInfra.
End-to-End Inventory Prediction and Contract Allocation for Guaranteed Delivery Advertising
Authors: Wuyang Mao, Chuanren Liu, Yundu Huang, Zhonglin Zu, M. Harshvardhan, Liang Wang 0001, Bo Zheng 0007
Venue: KDD
Year: 2023

Guaranteed Delivery (GD) advertising plays an essential part in e-commerce marketing, where the ad publisher signs contracts with advertisers in advance by promising delivery of advertising impressions to fulfill targeting requirements for advertisers. Previous research on GD advertising mainly focused on online serving yet overlooked the importance of contract allocation at the GD selling stage. Traditional GD selling approaches consider impression inventory prediction and contract allocation as two separate stages. However, such a two-stage optimization often leads to inferior contract allocation performance. In this paper, our goal is to reduce this performance gap with a novel end-to-end approach. Specifically, we propose the Neural Lagrangian Selling (NLS) model to jointly predict the impression inventory and optimize the contract allocation of advertising impressions with a unified learning objective. To this end, we first develop a differentiable Lagrangian layer to backpropagate the allocation problem through the neural network and allow direct optimization of the allocation regret. Then, for effective optimization with various allocation targets and constraints, we design a graph convolutional neural network to extract predictive features from the bipartite allocation graph. Extensive experiments show that our approach can improve GD selling performance compared with existing two-stage approaches. Particularly, our optimization layer can outperform the baseline solvers in both computational efficiency and solution quality. To the best of our knowledge, this is the first study to apply the end-to-end prediction and optimization approach for industrial GD selling problems. Our work has implications for general prediction and allocation problems as well.
Hands-on Tutorial: "Explanations in AI: Methods, Stakeholders and Pitfalls"
Authors: Mia C. Mayer, Muhammad Bilal Zafar, Luca Franceschi 0001, Huzefa Rangwala
Venue: KDD
Year: 2023

While using vast amounts of training data and sophisticated models has enhanced the predictive performance of Machine Learning (ML) and Artificial Intelligence (AI) solutions, it has also led to an increased difficulty in comprehending their predictions. The ability to explain predictions is often one of the primary desiderata for adopting AI and ML solutions [6, 13]. The desire for explainability has led to a rapidly growing body of literature on explainable AI (XAI) and has also resulted in the development of hundreds of XAI methods targeting different domains (e.g., finance, healthcare), applications (e.g., model debugging, actionable recourse), data modalities (e.g., tabular data, images), models (e.g., transformers, convolutional neural networks) and stakeholders (e.g., end-users, regulatory authorities, data scientists). The goal of this tutorial is to present a comprehensive overview of the XAI field to the participants. As a hands-on tutorial, we will showcase state-of-the-art methods that can be used for different data modalities and contexts to extract the right abstractions for interpretation. We will also cover common pitfalls when using explanations, e.g., misrepresentation, and lack of robustness of explanations.
Pretrained Language Representations for Text Understanding: A Weakly-Supervised Perspective
Authors: Yu Meng 0001, Jiaxin Huang 0001, Yu Zhang 0044, Yunyi Zhang 0001, Jiawei Han 0001
Venue: KDD
Year: 2023

Language representations pretrained on general-domain corpora and adapted to downstream task data have achieved enormous success in building natural language understanding (NLU) systems. While the standard supervised fine-tuning of pretrained language models (PLMs) has proven an effective approach for superior NLU performance, it often necessitates a large quantity of costly human-annotated training data. For example, the enormous success of ChatGPT and GPT-4 can be largely credited to their supervised fine-tuning with massive manually-labeled prompt-response training pairs. Unfortunately, obtaining large-scale human annotations is in general infeasible for most practitioners. To broaden the applicability of PLMs to various tasks and settings, weakly-supervised learning offers a promising direction to minimize the annotation requirements for PLM adaptions.
The 12th International Workshop on Urban Computing
Authors: Chuishi Meng, Yanhua Li, Yu Zheng 0004, Jieping Ye, Qiang Yang 0001, Philip S. Yu, Ouri Wolfson
Venue: KDD
Year: 2023

Urbanization's rapid progress has led to many big cities, which have modernized many people's lives but also engendered big challenges, such as air pollution, increased energy consumption and traffic congestion. Tackling these challenges were nearly impossible years ago given the complex and dynamic settings of cities. Nowadays, sensing technologies and large-scale computing infrastructures have produced a variety of big data in urban spaces, e.g., human mobility, air quality, traffic patterns, and geographical data. Motivated by the opportunities of building more intelligent cities, we came up with a vision of urban computing, which aims to unlock the power of knowledge from big and heterogeneous data collected in urban spaces and apply this powerful information to solve major issues our cities face today.
Hyper-USS: Answering Subset Query Over Multi-Attribute Data Stream
Authors: Ruijie Miao, Yiyao Zhang, Guanyu Qu, Kaicheng Yang 0001, Tong Yang 0003, Bin Cui 0001
Venue: KDD
Year: 2023

Sketching algorithms are considered as promising solutions for answering approximate query on massive data stream. In real scenarios, a large number of problems can be abstracted as subset query over multiple attributes. Existing sketches are designed for query on single attributes, and therefore are inefficient for query on multiple attributes. In this work, we propose Hyper-USS, an innovative sketching algorithm that supports subset query over multiple attributes accurately and efficiently. To the best of our knowledge, this work is the first sketching algorithm designed to answer approximate query over multi-attribute data stream. We utilize the key technique, Joint Variance Optimization, to guarantee high estimation accuracy on all attributes. Experiment results show that, compared with the state-of-the-art (SOTA) sketches that support subset query on single attributes, Hyper-USS improves the accuracy by 16.67X and the throughput by 8.54X. The code is open-sourced at Github.
Scenario-Adaptive Feature Interaction for Click-Through Rate Prediction
Authors: Erxue Min, Da Luo, Kangyi Lin, Chunzhen Huang, Yang Liu 0245
Venue: KDD
Year: 2023

Traditional Click-Through Rate (CTR) prediction models are usually trained and deployed in a single scenario. However, large-scale commercial platforms usually contain multiple recommendation scenarios, the traffic characteristics of which may be significantly different. Recent studies have proved that learning a unified model to serve multiple scenarios is effective in improving the overall performance. However, most existing approaches suffer from various limitations respectively, such as insufficient distinction modeling, inefficiency with the increase of scenarios, and lack of interpretability. More importantly, as far as we know, none of existing Multi-Scenario Modeling approaches takes explicit feature interaction into consideration when modeling scenario distinctions, which limits the expressive power of the network and thus impairs the performance. In this paper, we propose a novel Scenario-Adaptive Feature Interaction framework named SATrans, which models scenario discrepancy as the distinction of patterns in feature correlations. Specifically, SATrans is built on a Transformer architecture to learn high-order feature interaction and involves the scenario information in the modeling of self-attention to capture distribution shifts across scenarios. We provide various implementations of our framework to boost the performance, and experiments on both public and industrial datasets show that SATrans 1) significantly outperforms existing state-of-the-art approaches for prediction, 2) is parameter-efficient as the space complexity grows marginally with the increase of scenarios, 3) offers good interpretability in both instance-level and scenario-level. We have deployed the model in WeChat Official Account Platform and have seen more than 2.84% online CTR increase on average in three major scenarios.
Online Level-wise Hierarchical Clustering
Authors: Nicholas Monath, Manzil Zaheer, Andrew McCallum
Venue: KDD
Year: 2023

Online hierarchical clustering algorithms, compared to their scalable batch setting counterparts, typically provide more limited accuracy and efficiency performance. Yet, when data is incrementally arriving, a crucial setting in many clustering applications (e.g., entity resolution and concept discovery), these batch setting algorithms do not apply. This paper presents a family of new algorithms for online hierarchical clustering that combine high quality trees and fast per-point insertion time--made possible through a limited number of parallel non-greedy tree re-arrangements. We analyze our methods under assumptions about the data and the separability of clusters. Empirically, we find that our proposed algorithms yield state-of-the-art results in hierarchical clustering dendrogram purity and in building compressed prototypes for a k-nearest representative classifier.
Socially Responsible Machine Learning: A Causal Perspective
Authors: Raha Moraffah, Amir-Hossein Karimi, Adrienne Raglin, Huan Liu 0001
Venue: KDD
Year: 2023

The evergrowing reliance of humans and society on machine learning methods has raised concerns about their trustworthiness and liability. As a response to these concerns, Socially Responsible Machine Learning (SRML) aims at developing fair, transparent, and robust machine learning algorithms. However, traditional approaches to SRML do not incorporate human perspectives, and therefore are not sufficient to build long-lasting trust between machines and human being. Causality as the key to human intelligence plays a vital role in achieving socially responsible machine learning algorithms which are compatible with human notions. Bridging the gap between traditional SRML and causality, in this tutorial, we aim at providing a holistic overview of SRML through the lens of causality. In particular, we will focus on state-of-the-art techniques on causal socially responsible ML in terms of fairness, interpretability, and robustness. The objectives of this tutorial are as follows: (1) we provide a taxonomy of existing literature on causal socially responsible ML from fairness, interpretability, and robustness perspective; (2) we review the state-of-the-art techniques for each task; and (3) we elucidate open questions and future research directions. We believe this tutorial is beneficial to researchers and practitioners from the areas of data mining, machine learning, and social sciences.
Training Large-scale Foundation Models on Emerging AI Chips
Authors: Aashiq Muhamed, Christian Bock, Rahul Solanki, Youngsuk Park, Yida Wang 0003, Jun Huan
Venue: KDD
Year: 2023

Foundation models such as ChatGPT and GPT-4 have garnered significant interest from both academia and industry due to their emergent capabilities, such as few-shot prompting, multi-step reasoning, instruction following, and model calibration. Such capabilities were previously only attainable with specially designed models, such as those using knowledge graphs, but can now be achieved on a much larger scale with foundation models. As the capabilities of foundation models have increased, so too have their sizes at a rate much faster than Moore's law. For example, the BERT large model was initially released as a 334M model in 2018, and by 2023, the largest GPT-4 models are estimated to range between 200-300B, representing an increase of three orders of magnitude in just five years. The training of foundation models requires massive computing power. For instance, training a BERT model on a single state-of-the-art GPU machine with multi-A100 chips can take several days, while training GPT-3 models on a large multi-instance GPU cluster can take several months to complete the estimated 3 X 1023 flops.
Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications
Authors: Mila Nambiar, Supriyo Ghosh, Priscilla Ong, Yu En Chan, Yong Mong Bee, Pavitra Krishnaswamy
Venue: KDD
Year: 2023

There is increasing interest in data-driven approaches for recommending optimal treatment strategies in many chronic disease management and critical care applications. Reinforcement learning methods are well-suited to this sequential decision-making problem, but must be trained and evaluated exclusively on retrospective medical record datasets as direct online exploration is unsafe and infeasible. Despite this requirement, the vast majority of treatment optimization studies use off-policy RL methods (e.g., Double Deep Q Networks (DDQN) or its variants) that are known to perform poorly in purely offline settings. Recent advances in offline RL, such as Conservative Q-Learning (CQL), offer a suitable alternative. But there remain challenges in adapting these approaches to real-world applications where suboptimal examples dominate the retrospective dataset and strict safety constraints need to be satisfied. In this work, we introduce a practical and theoretically grounded transition sampling approach to address action imbalance during offline RL training. We perform extensive experiments on two real-world tasks for diabetes and sepsis treatment optimization to compare performance of the proposed approach against prominent off-policy and offline RL baselines (DDQN and CQL). Across a range of principled and clinically relevant metrics, we show that our proposed approach enables substantial improvements in expected health outcomes and in consistency with relevant practice and safety guidelines.
Causal Inference via Style Transfer for Out-of-distribution Generalisation
Authors: Toan Nguyen 0004, Kien Do, Duc Thanh Nguyen, Bao Duong, Thin Nguyen
Venue: KDD
Year: 2023

Out-of-distribution (OOD) generalisation aims to build a model that can generalise well on an unseen target domain using knowledge from multiple source domains. To this end, the model should seek the causal dependence between inputs and labels, which may be determined by the semantics of inputs and remain invariant across domains. However, statistical or non-causal methods often cannot capture this dependence and perform poorly due to not considering spurious correlations learnt from model training via unobserved confounders. A well-known existing causal inference method like back-door adjustment cannot be applied to remove spurious correlations as it requires the observation of confounders. In this paper, we propose a novel method that effectively deals with hidden confounders by successfully implementing front-door adjustment (FA). FA requires the choice of a mediator, which we regard as the semantic information of images that helps access the causal mechanism without the need for observing confounders. Further, we propose to estimate the combination of the mediator with other observed images in the front-door formula via style transfer algorithms. Our use of style transfer to estimate FA is novel and sensible for OOD generalisation, which we justify by extensive experimental results on widely used benchmark datasets.