OpenRaiser/Intern-Atlas huggingface dataset:
Pyramid Graph Neural Network: A Graph Sampling and Filtering Approach for Multi-scale Disentangled Representations
Authors: Haoyu Geng, Chao Chen 0016, Yixuan He 0001, Gang Zeng, Zhaobing Han, Hua Chai, Junchi Yan
Venue: KDD
Year: 2023

Spectral methods for graph neural networks (GNNs) have achieved great success. Despite their success, many works have shown that existing approaches are mainly focused on low-frequency information which may not be pertinent to the task at hand. Recent efforts have been made to design new graph filters for wider frequency profiles, but it remains an open problem how to learn multi-scale disentangled node embeddings in the graph Fourier domain. In this paper, we propose a graph (signal) sampling and filtering framework, entitled Pyramid Graph Neural Network (PyGNN), which follows the Downsampling-Filtering-Upsampling-Decoding scheme. To be specific, we develop an ω-bandlimited downsampling approach to split input graph into subgraphs for the reduction of high-frequency components, then perform spectral graph filters on subgraphs to achieve node embeddings with different frequency bands, and propose a Laplacian smoothing-based upsampling approach to extrapolate the node embedding on subgraphs to the full set of vertices on the original graph. In the end, we add frequency-aware gated units to decode node embeddings of different frequencies for downstream tasks. Results on both homophilic and heterophilic graph datasets show its superiority over state-of-the-art methods.
Addressing Bias and Fairness in Machine Learning: A Practical Guide and Hands-on Tutorial
Authors: Rayid Ghani, Kit T. Rodolfa, Pedro Saleiro, Sérgio M. Jesus
Venue: KDD
Year: 2023

As data science and machine learning (ML) increasingly shape our society, the importance of developing fair algorithmic decision-making systems becomes paramount. There is a pressing need to train data scientists and practitioners on handling bias and fairness in real-world scenarios, from early stages of a data science project to maintaining ML systems in production. Existing resources are mostly academic and cover the ML training and optimization aspects of bias mitigation, leaving practitioners without comprehensive frameworks for making decisions throughout a real-world project lifecycle. This tutorial aims to bridge the gap between research and practice, providing an in-depth exploration of algorithmic fairness, encompassing metrics and definitions, practical case studies, data bias understanding, bias mitigation and model fairness audits using the Aequitas toolkit. Participants will be equipped to engage in conversations about bias, assist decision-makers in understanding options and trade-offs, evaluate project scoping aspects influencing fairness outcomes, and define actions and interventions based on model predictions. They will also learn to identify cohorts, target variables, evaluation metrics, and establish bias and fairness goals for different groups. Moreover, participants will gain insights into auditing and mitigating model bias, and implementing continuous monitoring to assess retraining needs. The tutorial addresses the current lack of practical training materials, methodologies, and tools for researchers and developers working on real-world algorithmic decision-making systems. By the conclusion of this hands-on tutorial, attendees will be well-versed in navigating bias-related issues, selecting appropriate metrics, and applying bias audit and mitigation frameworks and tools for informed design decisions in real-world data science systems.
Privacy in Advertising: Analytics and Modeling
Authors: Badih Ghazi, Ravi Kumar 0001, Pasin Manurangsi
Venue: KDD
Year: 2023

Privacy in general, and differential privacy (DP) in particular, have become important topics in data mining and machine learning. Digital advertising is a critical component of the internet and is powered by large-scale data analytics and machine learning models; privacy concerns around these are on the rise. Despite the central importance of private ad analytics and training privacy-preserving ad prediction models, there has been relatively little exposure of this subject to the broader KDD community. In the past three years, the interest in privacy and the interest in online advertising have been steadily growing in KDD. The aim of this tutorial is to provide KDD researchers with an introduction to the problems that arise in private analytics and modeling in advertising, survey recent results, and describe the main research challenges in the space.
The Second Workshop on Applied Machine Learning Management
Authors: Dmitri Goldenberg, Chana Ross, Shir Meir Lador, Lin Lee Cheong, Panpan Xu, Elena Sokolova, Amit Mandelbaum, Irina Vasilinetc, Ankit Jain, Amit Weil Modlinger, Saloni Potdar
Venue: KDD
Year: 2023

Machine learning applications are rapidly adopted by industry leaders in any field. The growth of investment in AI-driven solutions created new challenges in managing Data Science and ML resources, people and projects as a whole. The discipline of managing applied machine learning teams, requires a healthy mix between agile product development tool-set and a long term research oriented mindset. The abilities of investing in deep research while at the same time connecting the outcomes to significant business results create a large knowledge based on management methods and best practices in the field. The Second KDD Workshop on Applied Machine Learning Management brings together applied research managers from various fields to share methodologies and case-studies on management of ML teams, products, and projects, achieving business impact with advanced AI-methods.
Sparse Binary Transformers for Multivariate Time Series Modeling
Authors: Matt Gorbett, Hossein Shirazi, Indrakshi Ray
Venue: KDD
Year: 2023

Compressed Neural Networks have the potential to enable deep learning across new applications and smaller computational environments. However, understanding the range of learning tasks in which such models can succeed is not well studied. In this work, we apply sparse and binary-weighted Transformers to multivariate time series problems, showing that the lightweight models achieve accuracy comparable to that of dense floating-point Transformers of the same structure. Our model achieves favorable results across three time series learning tasks: classification, anomaly detection, and single-step forecasting. Additionally, to reduce the computational complexity of the attention mechanism, we apply two modifications, which show little to no decline in model performance: 1) in the classification task, we apply a fixed mask to the query, key, and value activations, and 2) for forecasting and anomaly detection, which rely on predicting outputs at a single point in time, we propose an attention mask to allow computation only at the current time step. Together, each compression technique and attention modification substantially reduces the number of non-zero operations necessary in the Transformer. We measure the computational savings of our approach over a range of metrics including parameter count, bit size, and floating point operation (FLOPs) count, showing up to a 53× reduction in storage size and up to 10.5× reduction in FLOPs.
3D-Polishing for Triangular Mesh Compression of Point Cloud Data
Authors: Jiaqi Gu 0005, Guosheng Yin
Venue: KDD
Year: 2023

Triangular meshes are commonly used to reconstruct the surfaces of 3-dimensional (3D) objects based on the point cloud data. With an increasing demand for high-quality approximation, the sizes of point cloud data and the generated triangular meshes continue to increase, resulting in high computational cost in data processing, visualization, analysis, transmission and storage. Motivated by the process of sculpture polishing, we develop a novel progressive mesh compression approach, called the greedy 3D-polishing algorithm, to sequentially remove redundant points and triangles in a greedy manner while maintaining the approximation quality of the surface. Based on the polishing algorithm, we propose the approximate curvature radius to evaluate the scale of features polished at each iteration. By reformulating the compression rate selection as a change-point detection problem, a rank-based procedure is developed to select the optimal compression rate so that most of the global features of the 3D object surface can be preserved with statistical guarantee. Experiments on both moderate- and large-scale 3D shape datasets show that the proposed method can substantially reduce the size of the point cloud data and the corresponding triangular mesh, so that most of the surface information can be retained by a much smaller number of points and triangles.
Yggdrasil Decision Forests: A Fast and Extensible Decision Forests Library
Authors: Mathieu Guillame-Bert, Sebastian Bruch 0001, Richard Stotz, Jan Pfeifer
Venue: KDD
Year: 2023

Yggdrasil Decision Forests is a library for the training, serving and interpretation of decision forest models, targeted both at research and production work, implemented in C++, and available in C++, command line interface, Python (under the name TensorFlow Decision Forests), JavaScript, Go, and Google Sheets (under the name Simple ML for Sheets). The library has been developed organically since 2018 following a set of four design principles applicable to machine learning libraries and frameworks: simplicity of use, safety of use, modularity and high-level abstraction, and integration with other machine learning libraries. In this paper, we describe those principles in detail and present how they have been used to guide the design of the library. We then showcase the use of our library on a set of classical machine learning problems. Finally, we report a benchmark comparing our library to related solutions.
A Data-centric Framework to Endow Graph Neural Networks with Out-Of-Distribution Detection Ability
Authors: Yuxin Guo, Cheng Yang 0002, Yuluo Chen, Jixi Liu, Chuan Shi 0001, Junping Du 0001
Venue: KDD
Year: 2023

Out-of-distribution (OOD) detection, which aims to identify OOD samples from in-distribution (ID) ones in test time, has become an essential problem in machine learning. However, existing works are mostly conducted on Euclidean data, and the problem in graph-structured data remains under-explored. Several recent works begin to study graph OOD detection, but they all need to train a graph neural network (GNN) from scratch with high computational cost. In this work, we make the first attempt to endow a well-trained GNN with the OOD detection ability without modifying its parameters. To this end, we design a post-hoc framework with Adaptive Amplifier for Graph OOD Detection, named AAGOD, concentrating on data-centric manipulation. The insight of AAGOD is to superimpose a parameterized amplifier matrix on the adjacency matrix of each original input graph. The amplifier can be seen as prompts and is expected to emphasize the key patterns helpful for graph OOD detection, thereby enlarging the gap between OOD and ID graphs. Then well-trained GNNs can be reused to encode the amplified graphs into vector representations, and pre-defined scoring functions can further convert the representations into detection scores. Specifically, we design a Learnable Amplifier Generator (LAG) to customize amplifiers for different graphs, and propose a Regularized Learning Strategy (RLS) to train parameters with no OOD data required. Experiment results show that AAGOD can be applied on various GNNs to enable the OOD detection ability. Compared with the state-of-the-art baseline in graph OOD detection, on average AAGOD has 6.21% relative enhancement in AUC and a 34 times faster training speed. Code and data are available at https://github.com/BUPT-GAMMA/AAGOD.
SketchPolymer: Estimate Per-item Tail Quantile Using One Sketch
Authors: Jiarui Guo, Yisen Hong, Yuhan Wu 0001, Yunfei Liu, Tong Yang 0003, Bin Cui 0001
Venue: KDD
Year: 2023

1Estimating the quantile of distribution, especially tail distribution, is an interesting topic in data stream models, and has obtained extensive interest from many researchers. In this paper, we propose a novel sketch, namely SketchPolymer to accurately estimate per-item tail quantile. SketchPolymer uses a technique called Early Filtration to filter infrequent items, and another technique called VSS to reduce error. Our experimental results show that the accuracy of SketchPolymer is on average 32.67 times better than state-of-the-art techniques. We also implement our SketchPolymer on P4 and FPGA platforms to verify its deployment flexibility. All our codes are available at GitHub.[1]
Efficient Continuous Space Policy Optimization for High-frequency Trading
Authors: Li Han 0001, Nan Ding, Guoxuan Wang, Dawei Cheng, Yuqi Liang
Venue: KDD
Year: 2023

High-frequency trading is an extraordinarily intricate financial task, which is normally treated as a near real-time sequential decision problem. Compared with the traditional two-phase approach, forecasting equity's trend and then weighting them by combinatorial optimization, deep reinforcement learning (DRL) methods have shown advances in reward chasing with optimal policies. However, existing DRL-based methods either leverage portfolio optimization on low-frequency scenarios or only support a very limited number of assets with discrete action space, facing significant computing efficiency challenges. Therefore, we propose an efficient DRL-based policy optimization (DRPO) method for high-frequency trading. In particular, we model the portfolio management task with Markov Decision Process by directly inferring the equity weights in the action space guided by maximum accumulated returns. To reduce agents' interaction complexity without reducing interpretation, we detach the environment into the "static'' market states and "dynamic'' portfolio weight states. Then, we design an efficient reward expectation calculation algorithm via probabilistic dynamic programming, which enables our agents directly collect feedback away from trajectory sampling-based morass. To the best of our knowledge, this is the first work that solves the high-frequency portfolio optimization problem by devising an efficient continuous space policy optimization algorithm in the DRL framework. Through extensive experiments on the real-world data from Dow Jones, Coinbase and SSE exchanges, we show that our proposed DRPO significantly outperforms state-of-the-art benchmark methods. The results demonstrate the practical applicability and effectiveness of the proposed method.
iETA: A Robust and Scalable Incremental Learning Framework for Time-of-Arrival Estimation
Authors: Jindong Han, Hao Liu 0026, Shui Liu, Xi Chen 0080, Naiqiang Tan, Hua Chai, Hui Xiong 0001
Venue: KDD
Year: 2023

Time-of-arrival estimation or Estimated Time of Arrival (ETA) has become an indispensable building block of modern intelligent transportation systems. While many efforts have been made for time-of-arrival estimation, most of them have scalability and robustness issues when dealing with real-world large-scale ETA scenarios, where billions of vehicle trajectories and ETA requests have been continuously generating every day. To this end, in this paper, we propose a robust and scalable incremental ETA learning framework, iETA, to continuously exploit spatio-temporal traffic patterns from massive floating-car data and thus achieve better estimation performances. Specifically, we first build an incremental travel time predictor that can be incrementally updated based on newly generated traffic data. The incremental travel time predictor not only reduces the overall learning overhead but also improves the model's robustness toward urban traffic distribution shifts. Then, we propose a historical traffic knowledge consolidation module to preserve critical spatio-temporal knowledge from previous ETA predictors under the incremental learning setting. Moreover, to reduce interference induced by low-quality traffic data, we propose an adversarial training module to improve the learning robustness by proactively mitigating and resisting traffic noise perturbations. Finally, extensive experiments demonstrate the effectiveness and efficiency of the proposed system against state-of-the-art baselines in large-scale ETA scenarios. Most importantly, iETA has been deployed on the Didi Chuxing platform, handling real-time billions of ETA queries every day, and substantially improves the prediction accuracy.
Detecting Interference in Online Controlled Experiments with Increasing Allocation
Authors: Kevin Han, Shuangning Li, Jialiang Mao, Han Wu
Venue: KDD
Year: 2023

In the past decade, the technology industry has adopted online controlled experiments (a.k.a. A/B testing) to guide business decisions. In practice, A/B tests are often implemented with increasing treatment allocation: the new treatment is gradually released to an increasing number of units through a sequence of randomized experiments. In scenarios such as experimenting in a social network setting or in a bipartite online marketplace, interference among units may exist, which can harm the validity of simple inference procedures. In this work, we introduce a widely applicable procedure to test for interference in A/B testing with increasing allocation. Our procedure can be implemented on top of an existing A/B testing platform with a separate flow and does not require a priori a specific interference mechanism. In particular, we introduce two permutation tests that are valid under different assumptions. Firstly, we introduce a general statistical test for interference requiring no additional assumption. Secondly, we introduce a testing procedure that is valid under a time fixed effect assumption. The testing procedure is of very low computational complexity, it is powerful, and it formalizes a heuristic algorithm implemented already in industry. We demonstrate the performance of the proposed testing procedure through simulations on synthetic data. Finally, we discuss one application at LinkedIn, where a screening step is implemented to detect potential interference in all their marketplace experiments with the proposed methods in the paper.
GAT-MF: Graph Attention Mean Field for Very Large Scale Multi-Agent Reinforcement Learning
Authors: Qianyue Hao, Wenzhen Huang, Tao Feng, Jian Yuan, Yong Li 0008
Venue: KDD
Year: 2023

Recent advancements in reinforcement learning have witnessed remarkable achievements by intelligent agents ranging from game-playing to industrial applications. Of particular interest is the area of multi-agent reinforcement learning (MARL), which holds significant potential for real-world scenarios. However, typical MARL methods are limited in their ability to handle tens of agents, leaving scenarios with up to hundreds or even thousands of agents almost unexplored. The scaling up of the number of agents presents two primary challenges: (1) agent-agent interactions are crucial in multi-agent systems while the number of interactions grows quadratically with the number of agents, resulting in substantial computational complexity and difficulty in strategies-learning; (2) the strengths of interactions among agents exhibit variations both across agents and over time, making it difficult to precisely model such interactions. In this paper, we propose a novel approach named Graph Attention Mean Field (GAT-MF). By converting agent-agent interactions into interactions between each agent and a weighted mean field, we achieve a substantial reduction in computational complexity. The proposed method offers a precise modeling of interaction dynamics with mathematical proofs of its correctness. Additionally, we design a graph attention mechanism to automatically capture the diverse and time-varying strengths of interactions, ensuring an accurate representation of agent interactions. Through extensive experimentation conducted in both manual and real-world scenarios involving over 3000 agents, we validate the efficacy of our method. The results demonstrate that our method outperforms the best baseline method with a remarkable improvement of 42.7%. Furthermore, our method saves 86.4% training time and 19.2% GPU memory compared to the best baseline method. For reproducibility, our source codes and data are available at https://github.com/tsinghua-fib-lab/Large-Scale-MARL-GATMF.
Identifying Complicated Contagion Scenarios from Cascade Data
Authors: Galen Harrison, Amro Alabsi Aljundi, Jiangzhuo Chen, S. S. Ravi, Anil Kumar S. Vullikanti, Madhav V. Marathe, Abhijin Adiga
Venue: KDD
Year: 2023

We consider the setting of cascades that result from contagion dynamics on large realistic contact networks. We address the question of whether the structural properties of a (partially) observed cascade can characterize the contagion scenario and identify the interventions that might be in effect. Using epidemic spread as a concrete example, we study how social interventions such as compliance in social distancing, extent (and efficacy) of vaccination, and the transmissibility of disease can be inferred. The techniques developed are more generally applicable to other contagions as well.
CLUR: Uncertainty Estimation for Few-Shot Text Classification with Contrastive Learning
Authors: Jianfeng He, Xuchao Zhang, Shuo Lei, Abdulaziz Alhamadani, Fanglan Chen, Bei Xiao, Chang-Tien Lu
Venue: KDD
Year: 2023

Few-shot text classification has extensive application where the sample collection is expensive or complicated. When the penalty for classification errors is high, such as early threat event detection with scarce data, we expect to know "whether we should trust the classification results or reexamine them.'' This paper investigates the Uncertainty Estimation for Few-shot Text Classification (UEFTC), an unexplored research area. Given limited samples, a UEFTC model predicts an uncertainty score for a classification result, which is the likelihood that the classification result is false. However, many traditional uncertainty estimation models in text classification are unsuitable for implementing a UEFTC model. These models require numerous training samples, whereas the few-shot setting in UEFTC only provides a few or just one support sample for each class in an episode. We propose Contrastive Learning from Uncertainty Relations (CLUR) to address UEFTC. CLUR can be trained with only one support sample for each class with the help of pseudo uncertainty scores. Unlike previous works that manually set the pseudo uncertainty scores, CLUR self-adaptively learns them using our proposed uncertainty relations. Specifically, we explore four model structures in CLUR to investigate the performance of three common-used contrastive learning components in UEFTC and find that two of the components are effective. Experiment results prove that CLUR outperforms six baselines on four datasets, including an improvement of 4.52% AUPR on an RCV1 dataset in a 5-way 1-shot setting. Our code and data split for UEFTC are in https://github.com/he159ok/CLUR_UncertaintyEst_FewShot_TextCls.
Planning to Fairly Allocate: Probabilistic Fairness in the Restless Bandit Setting
Authors: Christine Herlihy, Aviva Prins, Aravind Srinivasan, John P. Dickerson
Venue: KDD
Year: 2023

Restless and collapsing bandits are often used to model budget-constrained resource allocation in settings where arms have action-dependent transition probabilities, such as the allocation of health interventions among patients. However, SOTA Whittle-index-based approaches to this planning problem either do not consider fairness among arms, or incentivize fairness without guaranteeing it. We thus introduce ProbFair, a probabilistically fair policy that maximizes total expected reward and satisfies the budget constraint while ensuring a strictly positive lower bound on the probability of being pulled at each timestep. We evaluate our algorithm on a real-world application, where interventions support continuous positive airway pressure (CPAP) therapy adherence among patients, as well as on a broader class of synthetic transition matrices. We find that ProbFair preserves utility while providing fairness guarantees.