OpenRaiser/Intern-Atlas huggingface dataset:
Adaptive Learning for Weakly Labeled Streams
Authors: Zhen-Yu Zhang, Yu-Yang Qian 0001, Yu-Jie Zhang, Yuan Jiang 0001, Zhi-Hua Zhou
Venue: KDD
Year: 2022

In plenty of real-world applications, data are collected in a streaming fashion, and their accurate labels are hard to obtain. For instance, in the environmental monitoring task, sensors are collecting the data all the time. Still, their labels are scarce because the labeling process requires human effort and can conceal annotation errors. This paper investigates the problem of learning with weakly labeled data streams, in which data are continuously collected, and only a limited subset of streaming data is labeled but potentially with noise. This setting is challenging and of great importance but rarely studied in the literature. When the data are constantly gathered with unknown noise on labels, it is quite challenging to design algorithms to obtain a well-generalized classifier. To address this difficulty, we propose a novel noise transition matrix estimation approach for data streams with scarce noisy labels by online anchor points identification. Based on that, we propose an adaptive learning algorithm for weakly labeled data streams via model reuse and effectively alleviate the negative influence of label noise with unlabeled data. Both theoretical analysis and extensive experiments justify and validate the effectiveness of the proposed approach.
Medical Symptom Detection in Intelligent Pre-Consultation Using Bi-directional Hard-Negative Noise Contrastive Estimation
Authors: Shiwei Zhang, Jichao Sun, Yu Huang, Xueqi Ding, Yefeng Zheng 0001
Venue: KDD
Year: 2022

Leveraging artificial intelligence (AI) techniques in medical applications is helping our world to deal with the shortage of healthcare workers and improve the efficiency and productivity of healthcare delivery. Intelligent pre-consultation (IPC) is a relatively new application deployed on mobile terminals for collecting patient's information before a face-to-face consultation. It takes advantages of state-of-the-art machine learning techniques to assist doctors on clinical decision-making. One of key functions of IPC is to detect medical symptoms from patient queries. By extracting symptoms from patient queries, IPC is able to collect more information on patient's health status by asking symptom-related questions. All collected information will be summarized as a medical record for doctors to make clinical decision. This saves a great deal of time for both doctors and patients. Detecting symptoms from patient's query is challenging, as most patients lack medical background and often tend to use colloquial language to describe their symptoms. In this work, we formulate symptom detection as a retrieval problem and propose a bi-directional hard-negative enforced noise contrastive estimation method (Bi-hardNCE) to tackle the symptom detection problem. Bi-hardNCE has both forward contrastive estimation and backward contrastive estimation, which forces model to distinguish the true symptom from negative symptoms and meanwhile distinguish true query from negative queries. To include more informative negatives, our Bi-hardNCE adopts a hard-negative mining strategy and a false-negative eliminating strategy, which achieved a significant improvement on performance. Our proposed model outperforms commonly used retrieval models by a large margin.
The 5th Artificial Intelligence of Things (AIoT) Workshop
Authors: Jian Zhang, Jian Tang 0008, Yiran Chen 0001, Jie Liu 0001, Jieping Ye, Marilyn Wolf, Vijaykrishnan Narayanan, Mani B. Srivastava, Michael I. Jordan, Victor Bahl
Venue: KDD
Year: 2022

With advancement of recent network and chip technologies, IoT devices are becoming smarter with increasing compute power, bandwidth, and storage available on the device. This enables intelligent decision making and information transferring on the devices and unleashes the power of AIoT (Artificial Intelligence of Things) that supports applications such as smart city/agriculture/manufacturing/health care and self-driving scenarios.
Sparx: Distributed Outlier Detection at Scale
Authors: Sean Zhang, Varun Ursekar, Leman Akoglu
Venue: KDD
Year: 2022

There is no shortage of outlier detection (OD) algorithms in the literature, yet a vast body of them are designed for a single machine. With the increasing reality of already cloud-resident datasets comes the need for distributed OD techniques. This area, however, is not only understudied but also short of public-domain implementations for practical use. This paper aims to fill this gap: We design Sparx, a data-parallel OD algorithm suitable for shared-nothing infrastructures, which we specifically implement in Apache Spark. Through extensive experiments on three real-world datasets, with several billions of points and millions of features, we show that existing open-source solutions fail to scale up; either by large number of points or high dimensionality, whereas Sparx yields scalable and effective performance. To facilitate practical use of OD on modern-scale datasets, we open-source Sparx under the Apache license at https://tinyurl.com/sparx2022.
Data-Driven Oracle Bone Rejoining: A Dataset and Practical Self-Supervised Learning Scheme
Authors: Chongsheng Zhang, Bin Wang 0063, Ke Chen 0004, Ruixing Zong, Bofeng Mo, Yi Men, George Almpanidis, Shanxiong Chen, Xiangliang Zhang 0001
Venue: KDD
Year: 2022

Oracle Bone Inscriptions (OBI) is one of the oldest scripts in the world. The rejoining of Oracle Bone (OB) fragments is of vital importance to the research of ancient scripts and history. Although significant progress has been achieved in the past decades, the rejoining work still heavily relies on domain knowledge and manual work, thus remains a low efficient and time-consuming process Therefore, an automatic and practical algorithm/system for OB rejoining is of great value to the OBI community. To this end, we collect a real-world dataset for rejoining Oracle Bone fragments, namely OB-Rejoin, which consists of 998 OB rubbing images that suffer from low quality image problems, due to intrinsic underground eroding over time and extrinsic imaging conditions in the past. Moreover, a practical Self-Supervised Splicing Network, S3-Net, is proposed to rejoin the OB fragments based on shape similarity of their borderlines. Specifically, we first transform the manually annotated borderline strokes of OB images into times series style shape representations, which are fed as input to a Generative Adversarial Network for augmenting positive pairs of rejoinable OBs for each OB fragment that does not have rejoinable counterparts. A Siamese network is trained on such augmented data in a contrastive learning manner to retrieve the matching OB fragments of an unseen query from an OB fragment gallery. Experiments on the OB-Rejoin benchmark show that our data-driven approach outperforms two recent methods for time-series analysis. In order to demonstrate its practical potential, we deploy the proposed S3-Net method in real tests and ultimately discover dozens of new rejoinings missed by domain experts for decades.
Few-shot Heterogeneous Graph Learning via Cross-domain Knowledge Transfer
Authors: Qiannan Zhang, Xiaodong Wu, Qiang Yang 0015, Chuxu Zhang, Xiangliang Zhang 0001
Venue: KDD
Year: 2022

Graph few-shot learning seeks to alleviate the label scarcity problem resulting from the difficulties and high cost of data annotations in graph learning. However, the overwhelming solutions in graph few-shot learning focus on homogeneous graphs, ignoring the ubiquitous heterogeneous graphs (HGs), which represent real-world complex systems and domain knowledge with multi-typed nodes interconnected by multi-typed edges. To this end, we study the cross-domain few-shot learning problem over HGs and develop a novel model for Cross-domain Heterogeneous Graph Meta learning (CrossHG-Meta). The general idea is to promote the HG node classification in the data-scarce target domain by transferring meta-knowledge from a series of HGs in data-rich source domains. The key challenges are to 1) combat the heterogeneity in HGs to acquire the transferable meta-knowledge; 2) handle the domain shifts between the source HG and target HG; and 3) fast adapt to novel target tasks with few-shot annotated examples. Regarding the graph heterogeneity, CrossHG-Meta firstly builds a graph encoder to aggregate heterogeneous neighborhood information from multiple semantic contexts. Secondly, to tackle domain shifts, a cross-domain meta-learning strategy is proposed to include a domain critic, which is designed to explicitly lead cross-domain adaptation for meta-tasks in different domains and improve model generalizability. Last, to further alleviate data scarcity, CrossHG-Meta leverages unlabelled information in source domains with auxiliary self-supervised learning task to provide cross-domain contrastive regularization alongside the meta-optimization process to facilitate node embedding. Extensive experimental results on three multi-domain HG datasets demonstrate that the proposed model outperforms various state-of-the-art baselines for multiple few-shot node classification tasks under the cross-domain setting.
1st ACM SIGKDD Workshop on Ethical Artificial Intelligence: Methods and Applications (EAI-KDD22)
Authors: Chen Zhao 0010, Feng Chen 0001, Xintao Wu, Christopher Funk, Anthony Hoogs
Venue: KDD
Year: 2022

Ethical AI has become increasingly important and it has been attracting attention from academia and industry, due to its increased popularity in real-world applications with fairness concerns. It also places fundamental importance on ethical considerations in determining legitimate and illegitimate uses of AI. Organizations that apply ethical AI have clearly stated well-defined review processes to ensure adherence to legal guidelines. Therefore, the wave of research at the intersection of ethical AI in data mining and machine learning has also influenced other fields of science, including computer vision, natural language processing, reinforcement learning, and social science. Despite these successes, ethical AI still faces many challenges. Consequently, there is an urgent need to bring experts and researchers together at prestigious venues to discuss ethical AI, which has been rarely seen in previous KDD conferences. This workshop will provide a premium platform for both research and industry from different backgrounds to exchange ideas on opportunities, challenges, and cutting-edge techniques in ethical AI.
MT-FlowFormer: A Semi-Supervised Flow Transformer for Encrypted Traffic Classification
Authors: Ruijie Zhao 0001, Xianwen Deng, Zhicong Yan, Jun Ma, Zhi Xue, Yijun Wang
Venue: KDD
Year: 2022

With the increasing demand for the protection of personal network meta-data, encrypted networks have grown in popularity, so do the challenge of monitoring and analyzing encrypted network traffic. Currently, some deep learning-based methods have been proposed to leverage statistical features for encrypted traffic classification, which are barely affected by encryption techniques. However, these works still suffer from two main intrinsic limitations: (1) the feature extraction process lacks a mechanism to take into account correlations between flows in the flow sequence; and (2) a large volume of manually-labeled data is required for training an effective deep classifier. In this paper, we propose a novel semi-supervised framework to address these problems. To be specific, an efficient classifier with attention mechanism is proposed to extract features from flow sequences with low computational cost. Then, a Mean Teacher-style semi-supervised framework is adopted to exploit the unlabeled traffic data, where a spatiotemporal data augmentation method is designed as the key component to explore the spatial and temporal relationship within the unlabeled traffic data. Experimental results on two real-world traffic datasets demonstrate that our method outperforms state-of-the-art methods with a large margin.
Adaptive Fairness-Aware Online Meta-Learning for Changing Environments
Authors: Chen Zhao 0010, Feng Mi, Xintao Wu, Kai Jiang 0002, Latifur Khan, Feng Chen 0001
Venue: KDD
Year: 2022

The fairness-aware online learning framework has arisen as a powerful tool for the continual lifelong learning setting. The goal for the learner is to sequentially learn new tasks where they come one after another over time and the learner ensures the statistic parity of the new coming task across different protected sub-populations (e.g. race and gender). A major drawback of existing methods is that they make heavy use of the i.i.d assumption for data and hence provide static regret analysis for the framework. However, low static regret cannot imply a good performance in changing environments where tasks are sampled from heterogeneous distributions. To address the fairness-aware online learning problem in changing environments, in this paper, we first construct a novel regret metric FairSAR by adding long-term fairness constraints onto a strongly adapted loss regret. Furthermore, to determine a good model parameter at each round, we propose a novel adaptive fairness-aware online meta-learning algorithm, namely FairSAOML, which is able to adapt to changing environments in both bias control and model precision. The problem is formulated in the form of a bi-level convex-concave optimization with respect to the model's primal and dual parameters that are associated with the model's accuracy and fairness, respectively. The theoretic analysis provides sub-linear upper bounds for both loss regret and violation of cumulative fairness constraints. Our experimental evaluation on different real-world datasets with settings of changing environments suggests that the proposed FairSAOML significantly outperforms alternatives based on the best prior online learning approaches.
Distributed Hybrid CPU and GPU training for Graph Neural Networks on Billion-Scale Heterogeneous Graphs
Authors: Da Zheng 0004, Xiang Song 0003, Chengru Yang, Dominique LaSalle, George Karypis
Venue: KDD
Year: 2022

Graph neural networks (GNN) have shown great success in learn- ing from graph-structured data. They are widely used in various applications, such as recommendation, fraud detection, and search. In these domains, the graphs are typically large and heterogeneous, containing many millions or billions of vertices and edges of different types. To tackle this challenge, we develop DistDGLv2, a system that extends DistDGL for training GNNs on massive heterogeneous graphs in a mini-batch fashion, using distributed hybrid CPU/GPU training. DistDGLv2 places graph data in distributed CPU memory and performs mini-batch computation in GPUs. For ease of use, DistDGLv2 adopts API compatible with Deep Graph Library (DGL)'s mini-batch training and heterogeneous graph API, which enables distributed training with almost no code modification. To ensure model accuracy, DistDGLv2 follows a synchronous training approach and allows ego-networks forming mini-batches to include non-local vertices. To ensure data locality and load balancing, DistDGLv2 partitions heterogeneous graphs by using a multi-level partitioning algorithm with min-edge cut and multiple balancing constraints. DistDGLv2 deploys an asynchronous mini- batch generation pipeline that makes computation and data access asynchronous to fully utilize all hardware (CPU, GPU, network, PCIe). We demonstrate DistDGLv2 on various GNN workloads. Our results show that DistDGLv2 achieves 2 - 3x speedup over DistDGL and 18× speedup over Euler. It takes only 5 - 10 seconds to complete an epoch on graphs with hundreds of millions of vertices on a cluster with 64 GPUs.
Contrastive Learning with Complex Heterogeneity
Authors: Lecheng Zheng, Jinjun Xiong, Yada Zhu, Jingrui He
Venue: KDD
Year: 2022

With the advent of big data across multiple high-impact applications, we are often facing the challenge of complex heterogeneity. The newly collected data usually consist of multiple modalities and are characterized with multiple labels, thus exhibiting the co-existence of multiple types of heterogeneity. Although state-of-the-art techniques are good at modeling the complex heterogeneity with sufficient label information, such label information can be quite expensive to obtain in real applications. Recently, researchers pay great attention to contrastive learning due to its prominent performance by utilizing rich unlabeled data. However, existing work on contrastive learning is not able to address the problem of false-negative pairs, i.e., some 'negative' pairs may have similar representations if they have the same label. To overcome the issues, in this paper, we propose a unified heterogeneous learning framework, which combines both the weighted unsupervised contrastive loss and the weighted supervised contrastive loss to model multiple types of heterogeneity. We first provide a theoretical analysis showing that the vanilla contrastive learning loss easily leads to the sub-optimal solution in the presence of false-negative pairs, whereas the proposed weighted loss could automatically adjust the weight based on the similarity of the learned representations to mitigate this issue. Experimental results on real-world data sets demonstrate the effectiveness and the efficiency of the proposed framework modeling multiple types of heterogeneity.
Combo-Fashion: Fashion Clothes Matching CTR Prediction with Item History
Authors: Chenxu Zhu, Peng Du, Weinan Zhang 0001, Yong Yu 0001, Yang Cao
Venue: KDD
Year: 2022

As one of the fundamental trends for future development of recommender systems, Fashion Clothes Matching Recommendation for click-through rate (CTR) prediction has become an increasingly essential task. Unlike traditional single-item recommendation, a combo item, composed of a top item (e.g. a shirt) and a bottom item (e.g. a skirt), is recommended. In such a task, the matching effect between these two single items plays a crucial role, and greatly influences the users' preferences; however, it is usually neglected by previous approaches in CTR prediction. In this work, we tackle this problem by designing a novel algorithm called Combo-Fashion, which extracts the matching effect by introducing the matching history of the combo item with two cascaded modules: (i) Matching Search Module (MSM) seeks the popular combo items and undesirable ones as a positive set and a negative set, respectively; (ii) Matching Prediction Module (MPM) models the precise relationship between the candidate combo item and the positive/negative set by an attention-based deep model. Besides, the CPM Fashion Attribute, considered from characteristic, pattern and material, is applied to capture the matching effect further. As part of this work, we release two large-scale datasets consisting of 3.56 million and 6.01 million user behaviors with rich context and fashion information in millions of combo items. The experimental results over these two real-world datasets have demonstrated the superiority of our proposed model with significant improvements. Furthermore, we have deployed Combo-Fashion onto the platform of Taobao to recommend the combo items to the users, where an 8-day online A/B test proved the effectiveness of Combo-Fashion with an improvement of pCTR by 1.02% and uCTR by 0.70%.
User-tag Profile Modeling in Recommendation System via Contrast Weighted Tag Masking
Authors: Chenxu Zhu, Peng Du, Xianghui Zhu, Weinan Zhang 0001, Yong Yu 0001, Yang Cao
Venue: KDD
Year: 2022

User-tag profile modeling has become one of the novel and significant trends for the future development of industrial recommendation systems, which can be divided into two fundamental tasks: User Preferred Tag (UPT) and Tag Preferred User (TPU) in practical scenarios. In most existing deep learning models for user-tag profiling, the network inputs all the combined tags of the item with the user features when training but inputs only one tag with the user feature to evaluate the user's preference on a single tag when testing. This leads to data discrepancy between the training and testing samples. To address such an issue, we attempt a novel Random Masking Model (RMM) to remain only one tag at the training time by masking. However, it causes two other serious downsides. First, not all tags attached to the same item are equally predictive. Irrelevant tags may introduce noisy signals and thus cause performance degradation. Second, it neglects the impact of combined tags aggregated together, which may be an essential factor leading to user clicks. Therefore, we further propose a framework called Contrast Weighted Tag Masking (CWTM) in this work, which tackles these two issues with two modules: (i) Weighted Masking Module (WMM) introduces the importance network to compute a score for each tag attached to the item and then samples from these tags weightedly according to the score; (ii) Contrast Module (CM) makes use of a contrastive learning architecture to inherit and distill some understanding about the effect of aggregated tags. Offline experiments on four datasets (three public datasets and one proprietary industrial dataset) demonstrate the superiority and effectiveness of CWTM over the state-of-the-art baselines. Moreover, CWTM has been deployed on the training platform of Alibaba advertising systems and achieved substantial improvements of ROI and CVR by 16.8% and 9.6%, respectively.
How does Heterophily Impact the Robustness of Graph Neural Networks?: Theoretical Connections and Practical Implications
Authors: Jiong Zhu, Junchen Jin, Donald Loveland, Michael T. Schaub, Danai Koutra
Venue: KDD
Year: 2022

We bridge two research directions on graph neural networks (GNNs), by formalizing the relation between heterophily of node labels (i.e., connected nodes tend to have dissimilar labels) and the robustness of GNNs to adversarial attacks. Our theoretical and empirical analyses show that for homophilous graph data, impactful structural attacks always lead to reduced homophily, while for heterophilous graph data the change in the homophily level depends on the node degrees. These insights have practical implications for defending against attacks on real-world graphs: we deduce that separate aggregators for ego- and neighbor-embeddings, a design principle which has been identified to significantly improve prediction for heterophilous graph data, can also offer increased robustness to GNNs. Our comprehensive experiments show that GNNs merely adopting this design achieve improved empirical and certifiable robustness compared to the best-performing unvaccinated model. Additionally, combining this design with explicit defense mechanisms against adversarial attacks leads to an improved robustness with up to 18.33% performance increase under attacks compared to the best-performing vaccinated model.