OpenRaiser/Intern-Atlas huggingface dataset:
Fragility Index: A New Approach for Binary Classification
Authors: Chen Yang 0025, Ziqiang Zhang, Bo Cao 0007, Zheng Cui, Bin Hu, Tong Li, Daniel Zhuoyu Long, Jin Qi, Feng Wang 0023, Ruohan Zhan
Venue: KDD
Year: 2023

In binary classification problems, many performance metrics evaluate the probability that some error exceeds a threshold. Nevertheless, they focus more on the probability and fail to capture the magnitude of the error, which evaluates how large this error exceeds the threshold. Capturing the magnitude of error is desired in many applications. For example, in detecting disease and predicting credit default, the magnitude of error illustrates the confidence in making the wrong prediction. We propose a novel metric, the Fragility Index (FI), to evaluate the performance of binary classifiers by capturing the magnitude of the error. FI alleviates the risk of misclassification by penalizing the large error greatly, which is seldom considered by standard metrics. Moreover, to strengthen the generalization ability and handle unseen samples, we adopt the framework of distributionally robust optimization and robust satisficing, which allows us to derive and control the maximum degree of fragility of the classifier when the distribution of samples shifts. We show that FI can be easily calculated and optimized for common probabilistic distance measures. Experiments with real datasets demonstrate the new insights brought by FI and the advantages of classifiers selected under FI, which always improve the robustness and reduce the risk of large errors as compared to classifiers selected by alternative metrics.
From Labels to Decisions: A Mapping-Aware Annotator Model
Authors: Evan Yao, Jagdish Ramakrishnan, Xu Chen 0046, Viet-An Nguyen, Udi Weinsberg
Venue: KDD
Year: 2023

Online platforms regularly rely on human annotators to make real-time operational decisions for tasks such as content moderation. While crowdsourcing models have been proposed for aggregating noisy labels, they do not generalize well when annotators produce a labels in a large space, e.g., generated from complex review trees. We study a novel crowdsourcing setting with D possible operational decisions or outcomes, but annotators produce labels in a larger space of size L > D which are mapped to decisions through a known mapping function. For content moderation, such labels can correspond to violation reasons (e.g. nudity, violence), while the space of decisions is binary: remove the content or keep it up. In this setting, it is more important to make the right decision rather than estimating the correct underlying label. Existing methods typically separate out the labels to decisions mapping from the modeling of annotators, leading to sub-optimal statistical inference efficiency and excessive computation complexity. We propose a novel confusion matrix model for each annotator that leverages this mapping. Our model is parameterized in a hierarchical manner with both population parameters shared across annotators to model shared confusions and individual parameters to admit heterogeneity among annotators. With extensive numerical experiments, we demonstrate that the proposed model substantially improves accuracy over existing methods and scales well for moderate and large L. In a real-world application on content moderation at Meta, the proposed method offers a 13% improvement in AUC over prior methods, including Meta's existing model in production.
PAT: Geometry-Aware Hard-Label Black-Box Adversarial Attacks on Text
Authors: Muchao Ye, Jinghui Chen, Chenglin Miao, Han Liu 0008, Ting Wang 0006, Fenglong Ma
Venue: KDD
Year: 2023

Despite a plethora of prior explorations, conducting text adversarial attacks in practical settings is still challenging with the following constraints: black box -- the inner structure of the victim model is unknown; hard label -- the attacker only has access to the top-1 prediction results; and semantic preservation - the perturbation needs to preserve the original semantics. In this paper, we present PAT, a novel adversarial attack method employed under all these constraints. Specifically, PAT explicitly models the adversarial and non-adversarial prototypes and incorporates them to measure semantic changes for replacement selection in the hard-label black-box setting to generate high-quality samples. In each iteration, PAT finds original words that can be replaced back and selects better candidate words for perturbed positions in a geometry-aware manner guided by this estimation, which maximally improves the perturbation construction and minimally impacts the original semantics. Extensive evaluation with benchmark datasets and state-of-the-art models shows that PAT outperforms existing text adversarial attacks in terms of both attack effectiveness and semantic preservation. Moreover, we validate the efficacy of PAT against industry-leading natural language processing platforms in real-world settings.
VQNE: Variational Quantum Network Embedding with Application to Network Alignment
Authors: Xinyu Ye, Ge Yan 0001, Junchi Yan
Venue: KDD
Year: 2023

Learning of network embedding with vector-based node representation has attracted wide attention over the decade. It differs from the general setting of graph node embedding whereby the node attributes are also considered and yet may incur privacy issues. In this paper, we depart from the classic CPU/GPU architecture to consider the well-established network alignment problem based on network embedding, and develop a quantum machine learning approach with a low qubit cost for its near-future applicability on Noisy Intermediate-Scale Quantum (NISQ) devices. Specifically, our model adopts the discrete-time quantum walk (QW) and conducts the QW on the tailored merged network to extract structure information from the two aligning networks without the need for quantum state preparation which otherwise requires high quantum gate cost. Then the quantum states from QW are fed to a quantum embedding ansatz (i.e., parameterized circuit) to learn the latent representation of each node. The key part of our approach is to connect these two quantum modules to achieve a pure quantum paradigm without involving classical modules. To our best knowledge, there has not been any classic-quantum hybrid approach to network embedding, let alone a pure quantum paradigm being free from the bottleneck of communication between classic devices and quantum devices, which is still an open problem. Experimental results on two real-world datasets show the effectiveness of our quantum embedding approach in comparison with classical embedding approaches. Our model is readily and efficiently implemented in Python with a full-amplitude simulation of the QW and the quantum circuit. Therefore, our model can be readily deployed on an existing NISQ device with all the circuits provided, and only 13 qubits are needed in the experiments, which is rarely attained in existing quantum graph learning works.
Self-supervised Classification of Clinical Multivariate Time Series using Time Series Dynamics
Authors: Yakir Yehuda, Daniel Freedman, Kira Radinsky
Venue: KDD
Year: 2023

To improve the accuracy of clinical multivariate time series (MTS) classification (such as EEG and ECG) by a novel self-supervised paradigm that directly captures the dynamics between the different time series learned together to optimize the classification task. Labels in clinical datasets are very often insufficient. One way to address this challenge is leveraging self-supervision. This paradigm attempts to identify a supervisory signal inherent within a dataset to serve as a surrogate label. We present a novel form of self-supervision: dynamics of clinical MTS. Unlike other self-supervision methods, such as masking, that are intuitive but still heuristic, we suggest to learn a representation justified by Koopman theory. The latter was shown useful for representing clinical time series and can be used as a form of surrogate task to improve the clinical MTS classification. In the ECG task, we show that our proposed framework achieved higher sensitivity and specificity than the state-of-the-art (SOTA) baseline over numerous common diagnoses. For EEG abnormality classification, our proposed framework also achieved higher sensitivity and specificity than the SOTA baseline. All results are statistically significant. Our technique yields reliable clinical diagnosis in an empirical study employing signals from thousands of patients in multiple clinical tasks employing two types of clinical-grade sensors (ECG and EEG) as compared to the state-of-the-art machine learning. Leveraging time-series-dynamics self-supervision can help mitigate the lack of labels in clinical datasets used for training machine learning algorithms and significantly improve their performance. Specifically, the ECG system presented in this work is being trialed in hospitals, used by top cardiologists for patient diagnosis and treatment. We believe that the deployment of such cutting-edge technology will significantly improve the accuracy and speed of cardiac assessments.
DGI: An Easy and Efficient Framework for GNN Model Evaluation
Authors: Peiqi Yin, Xiao Yan 0002, Jinjing Zhou, Qiang Fu, Zhenkun Cai, James Cheng, Bo Tang 0016, Minjie Wang
Venue: KDD
Year: 2023

While many systems have been developed to train graph neural networks (GNNs), efficient model evaluation, which computes node embedding according to a given model, remains to be addressed. For instance, using the widely adopted node-wise approach, model evaluation can account for over 90% of the time in the end-to-end training process due to neighbor explosion, which means that a node accesses its multi-hop neighbors. The layer-wise approach avoids neighbor explosion by conducting computation layer by layer in GNN models. However, layer-wise model evaluation takes considerable implementation efforts because users need to manually decompose the GNN model into layers, and different implementations are required for GNN models with different structures.
Learning Multivariate Hawkes Process via Graph Recurrent Neural Network
Authors: Kanghoon Yoon, Youngjun Im, Jingyu Choi, Taehwan Jeong, Jinkyoo Park
Venue: KDD
Year: 2023

This paper presents a novel approach for modeling and predicting patterns of events in time-series learning, named graph recurrent temporal point process (GRTPP). Prior research has focused on using deep learning techniques, such as recurrent neural networks (RNNs) or attention-based sequential data embedding, on modeling the time-varying intensity of events. However, these models were typically limited to modeling a single intensity function capturing the event occurrence of all event types simultaneously. GRTPP addresses this issue by encoding multivariate event sequences into a sequence of graphs, where each node contains information about the event occurrence and time. The sequence of graphs is then embedded into node embeddings for each event type, taking into account the relationships between the event types. By integrating the estimated intensity functions, GRTPP predicts the event type and the timing of the next event. The proposed GRTPP model offers improved effectiveness and explainability compared to previous models, as demonstrated through empirical evaluations on five real-world datasets and the actual credit card transaction dataset. The code is available at https://github.com/im0j/GRTPP https://github.com/im0j/GRTPP.
The Second Workshop on Knowledge-Augmented Methods for Natural Language Processing
Authors: Wenhao Yu 0002, Lingbo Tong, Weijia Shi, Nanyun Peng 0001, Meng Jiang 0001
Venue: KDD
Year: 2023

Language models are being developed and deployed in many applications, "small"-scale and large-scale, generic and specialized, text-only and multimodal, etc. Meanwhile, the missingness of important knowledge causes limitations and safety challenges. The knowledge includes commonsense, world facts, domain expertise, personalization, and especially the unique patterns that need to be discovered from big data applications. Training and inference processes of the language models can be and should be augmented with the knowledge. The first KnowledgeNLP at AAAI 2023 attracted scientists on knowledge augmentation methods towards higher language intelligence. This workshop offers a broad platform to share ideas and discuss various topics, such as (1) synergy between knowledge and language model, (2) scalable architectures that integrate NLP, knowledge graph, and graph learning technologies, (3) KnowledgeNLP for e-commerce, education, and healthcare, (4) human factors and social good in KnowledgeNLP.
Group-based Fraud Detection Network on e-Commerce Platforms
Authors: Jianke Yu, Hanchen Wang 0001, Xiaoyang Wang 0002, Zhao Li 0007, Lu Qin 0001, Wenjie Zhang 0001, Jian Liao 0001, Ying Zhang 0001
Venue: KDD
Year: 2023

Along with the rapid technological and commercial innovation on the e-commerce platforms, there are an increasing number of frauds that bring great harm to these platforms. Many frauds are conducted by organized groups of fraudsters for higher efficiency and lower costs, which are also known as group-based frauds. Despite the high concealment and strong destructiveness of group-based fraud, there is no existing research work that can thoroughly exploit the information within the transaction networks of e-commerce platforms for group-based fraud detection. In this work, we analyze and summarize the characteristics of group-based frauds, based on which we propose a novel end-to-end semi-supervised Group-based Fraud Detection Network (GFDN) to support such fraud detection in real-world applications. Experimental results on large-scale e-commerce datasets from Taobao and Bitcoin trading datasets show the superior effectiveness and efficiency of our proposed model for group-based fraud detection on bipartite graphs.
Cognitive Evolutionary Search to Select Feature Interactions for Click-Through Rate Prediction
Authors: Runlong Yu, Xiang Xu, Yuyang Ye 0002, Qi Liu 0003, Enhong Chen
Venue: KDD
Year: 2023

Click-Through Rate (CTR) prediction of intelligent marketing systems is of great importance, in which feature interaction selection plays a key role. Most approaches model interactions of features by the same pre-defined operation under expert guidance, among which improper interactions may bring unnecessary noise and complicate the training process. To that end, in this paper, we aim to adaptively evolve the model to select proper operations to interact on feature pairs under task guidance. Inspired by natural evolution, we propose a general Cognitive EvoLutionary Search (CELS) framework, where cognitive ability refers to the malleability of organisms to orientate to the environment. Specifically, we conceptualize interactions as genomes, models as organisms, and tasks as natural environments. Mirroring how genetic malleability develops environmental adaptability, we thus diagnose the fitness of models to simulate the survival rates of organisms for natural selection, thereby an evolution path can be planned and visualized, offering an intuitive interpretation of the mechanisms underlying interaction modeling and selection. Based on the CELS framework, we develop four instantiations including individual-based search and population-based search. We demonstrate how individual mutation and population crossover enable CELS to evolve into diverse models suitable for various tasks and data, providing ready-to-use models. Extensive experiments on real-world datasets demonstrate that CELS significantly outperforms state-of-the-art approaches.
Multi Datasource LTV User Representation (MDLUR)
Authors: Junwoo Yun, Wonryeol Kwak, Joohyun Kim
Venue: KDD
Year: 2023

In this paper, we propose a novel user representation methodology called Multi Datasource LTV User Representation (MDLUR). Our model aims to establish a universal user embedding for downstream tasks, specifically lifetime value (LTV) prediction on specific days after installation. MDLUR uses a combination of various data sources, including user information, portrait, and behavior data from the first n days after installation of the social casino game "Club Vegas Slots" developed by Bagelcode. This model overcomes the limitation of conventional approaches that struggle with effectively utilizing various data sources or accurately capturing interactions in sparse datasets. MDLUR adopts unique model architectures tailored to each data source. Coupled with robust dimensionality reduction techniques, this model succeeds in the effective integration of insights from various data sources. Comprehensive experiments on real-world industrial data demonstrate the superiority of the proposed methods compared to SOTA baselines including Two-Stage XGBoost, WhalesDector, MSDMT, and BST. Not only did it outperform these models, but it has also been efficiently deployed and tested in a live environment using MLOps demonstrating its maintainability. The representation may potentially be applied to a wide range of downstream tasks, including conversion, churn, and retention prediction, as well as user segmentation and item recommendation.
Commonsense Knowledge Graph towards Super APP and Its Applications in Alipay
Authors: Xiaoling Zang, Binbin Hu, Jun Chu, Zhiqiang Zhang 0012, Guannan Zhang, Jun Zhou 0011, Wenliang Zhong
Venue: KDD
Year: 2023

The recently explosive growth of Super Apps brings great convenience to people's daily life by providing a wide variety of services through mini-programs, including online shopping, travel, finance, and so on. Due to the considerable gap between various scenarios, the restriction of effective information transfer and sharing severely blocks the efficient delivery of online services, potentially affecting the user's app experience. To deeply understand users' needs, we propose SupKG, a commonsense knowledge graph towards Super APP to help comprehensively characterize user behaviors across different business scenarios. In particular, our SupKG is carefully established from multiplex and heterogeneous data source in Alipay (a well-known Super App in China), which also emphasize abundant spatiotemporal relations and intent-related entities to answer the fundamental question in life service ''which service do users need at what time and where''.
Mining Electronic Health Records for Real-World Evidence
Authors: Chengxi Zang, Weishen Pan, Fei Wang 0001
Venue: KDD
Year: 2023

The rapid accumulation of large-scale Electronic Health Records (EHR) presents considerable opportunities to generate real-world evidence to inform clinical decision-making and accelerate drug development. However, the complexity of EHR has turned them into a formidable testing ground for cutting-edge AI algorithms. Furthermore, a significant gap still exists between algorithm development in the computer science community and clinical translation within the healthcare community. This tutorial aims to bridge this divide by fostering mutual understanding between the two communities by discussing using advanced machine learning and data mining technologies tailored to tackle real-world healthcare challenges, including 1) using EHR and trial emulation for understanding Long Covid and drug repurposing for Alzheimer's disease, and 2) risk prediction and associated fairness, interpretability, generalizability, etc., issues. We will conclude this tutorial by delving into potential opportunities for future research and unveiling the prospects of a career as a health data scientist.
HiMacMic: Hierarchical Multi-Agent Deep Reinforcement Learning with Dynamic Asynchronous Macro Strategy
Authors: Hancheng Zhang, Guozheng Li 0002, Chi Harold Liu, Guoren Wang, Jian Tang 0008
Venue: KDD
Year: 2023

Multi-agent deep reinforcement learning (MADRL) has been widely used in many scenarios such as robotics and game AI. However, existing methods mainly focus on the optimization of agents' micro policies without considering the macro strategy. As a result, they cannot perform well in complex or sparse reward scenarios like the StarCraft Multi-Agent Challenge (SMAC) and Google Research Football (GRF). To this end, we propose a hierarchical MADRL framework called "HiMacMic" with dynamic asynchronous macro strategy. Spatially, HiMacMic determines a critical position by using a positional heat map. Temporally, the macro strategy dynamically decides its deadline and updates it asynchronously among agents. We validate HiMacMic in four widely used benchmarks, namely: Overcooked, GRF, SMAC and SMAC-v2 with nine chosen scenarios. Results show that HiMacMic not only converges faster and achieves higher results than ten existing approaches, but also shows its adaptability to different environment settings.
Understanding the Semantics of GPS-based Trajectories for Road Closure Detection
Authors: Jiasheng Zhang, Kaiqiang An, Guoping Liu, Xiang Wen, Runbo Hu, Jie Shao 0001
Venue: KDD
Year: 2023

The accurate detection of road closures is of great value for real-time updating of digital maps. The existing methods mainly follow the paradigm of detecting the drastic changes in traffic statistical values (e.g., traffic flow), but they may lead to misidentifying since 1) drastic changes of traffic statistical values are hard to be observed in low-heat roads where the passing vehicles are sparse; 2) statistical values are sensitive to noise (e.g., traffic flow for tiny roads and tunnels is prone to miscounting); and 3) statistical values are naturally delayed, and misidentifying may occur when they have not yet shown significant changes. Surprisingly, since GPS-based trajectories can also exhibit significant abnormal patterns for road closures and have the superiority in fine granularity and timeliness, they can naturally tackle the above challenges. In this paper, we present a novel road closure detection framework based on mining the semantics of trajectories, called T-Closure. We first construct a heterogeneous graph based on the trajectory and the planned route to extract the spatial-topological property of each trajectory, where a node-level auxiliary task is proposed to guide the learning of feature encoders. A multi-view heterogeneous graph neural network (MVH-GNN) with a graph-level auxiliary task is then introduced to capture the semantics of trajectories, where intra-category relevance and inter-category interaction are both considered. Finally, a sequence-level auxiliary task refines the ability of LSTM in modeling the semantic relevance among trajectories while enhancing the robustness of our framework. Experiments on four real-world road closure datasets demonstrate the superiority of T-Closure. Online performance shows that T-Closure can detect 7000+ closure events monthly, with a delay within 1.5 hours.