OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic Network Anomaly Modeling of Cell-Phone Call Detail Records for Infectious Disease Surveillance
Authors: Carl Yang 0001, Hongwen Song, Mingyue Tang, Leon Danon, Ymir Vigfusson
Venue: KDD
Year: 2022

Global monitoring of novel diseases and outbreaks is crucial for pandemic prevention. To this end, movement data from cell-phones is already used to augment epidemiological models. Recent work has posed individual cell-phone metadata as a universal data source for syndromic surveillance for two key reasons: (1) these records are already collected for billing purposes in virtually every country and (2) they could allow deviations from people's routine behaviors during symptomatic illness to be detected, both in terms of mobility and social interactions. In this paper, we develop the necessary models to conduct population-level infectious disease surveillance by using cell-phone metadata individually linked with health outcomes. Specifically, we propose GraphDNA---a model that builds Graph neural networks (GNNs) into Dynamic Network Anomaly detection. Using cell-phone call records (CDR) linked with diagnostic information from Iceland during the H1N1v influenza outbreak, we show that GraphDNA outperforms state-of-the-art baselines on individual Date-of-Diagnosis (DoD) prediction, while tracking the epidemic signal in the overall population. Our results suggest that proper modeling of the universal CDR data could inform public health officials and bolster epidemic preparedness measures.
Learning Classifiers under Delayed Feedback with a Time Window Assumption
Authors: Shota Yasui, Masahiro Kato
Venue: KDD
Year: 2022

We consider training a binary classifier under delayed feedback (DF learning). For example, in the conversion prediction in online ads, we initially receive negative samples that clicked the ads but did not buy an item; subsequently, some samples among them buy an item then change to positive. In the setting of DF learning, we observe samples over time, then learn a classifier at some point. We initially receive negative samples; subsequently, some samples among them change to positive. This problem is conceivable in various real-world applications such as online advertisements, where the user action takes place long after the first click. Owing to the delayed feedback, naive classification of the positive and negative samples returns a biased classifier. One solution is to use samples that have been observed for more than a certain time window assuming these samples are correctly labeled. However, existing studies reported that simply using a subset of all samples based on the time window assumption does not perform well, and that using all samples along with the time window assumption improves empirical performance. We extend these existing studies and propose a method with the unbiased and convex empirical risk that is constructed from all samples under the time window assumption. To demonstrate the soundness of the proposed method, we provide experimental results on a synthetic and open dataset that is the real traffic log datasets in online advertising.
Multilingual Taxonomic Web Page Classification for Contextual Targeting at Yahoo
Authors: Eric Ye, Xiao Bai 0002, Neil O&apos, Hare, Eliyar Asgarieh, Kapil Thadani, Francisco Perez-Sorrosal, Sujyothi Adiga
Venue: KDD
Year: 2022

As we move toward a cookie-less world, the ability to track users' online activities for behavior targeting will be drastically reduced, making contextual targeting an appealing alternative for advertising platforms. Category-based contextual targeting displays ads on web pages that are relevant to advertiser-targeted categories, according to a pre-defined taxonomy. Accurate web page classification is key to the success of this approach. In this paper, we use multilingual Transformer-based transfer learning models to classify web pages in five high-impact languages. We adopt multiple data sampling techniques to increase coverage for rare categories, and modify the loss using class-based re-weighting to smooth the influence of frequent versus rare categories. Offline evaluation shows that these are crucial for improving our classifiers. We leverage knowledge distillation to train accurate models that are lightweight in terms of (i) model size, and (ii) the input text used. Classifying web pages using only text from the URL addresses a unique challenge for contextual targeting in that bid requests come to ad systems as URLs without content, while crawling is time consuming and costly. We launched the proposed models for contextual targeting in the Yahoo DSP, significantly increasing its revenue.
Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic Treatment Regimes
Authors: Changchang Yin, Ruoqi Liu, Jeffrey M. Caterino, Ping Zhang 0016
Venue: KDD
Year: 2022

Despite intense efforts in basic and clinical research, an individualized ventilation strategy for critically ill patients remains a major challenge. Recently, dynamic treatment regime (DTR) with reinforcement learning (RL) on electronic health records (EHR) has attracted interest from both the healthcare industry and machine learning research community. However, most learned DTR policies might be biased due to the existence of confounders. Although some treatment actions non-survivors received may be helpful, if confounders cause the mortality, the training of RL models guided by long-term outcomes (e.g., 90-day mortality) would punish those treatment actions causing the learned DTR policies to be suboptimal. In this study, we develop a new deconfounding actor-critic network (DAC) to learn optimal DTR policies for patients. To alleviate confounding issues, we incorporate a patient resampling module and a confounding balance module into our actor-critic framework. To avoid punishing the effective treatment actions non-survivors received, we design a short-term reward to capture patients' immediate health state changes. Combining short-term with long-term rewards could further improve the model performance. Moreover, we introduce a policy adaptation method to successfully transfer the learned model to new-source small-scale datasets. The experimental results on one semi-synthetic and two different real-world datasets show the proposed model outperforms the state-of-the-art models. The proposed model provides individualized treatment decisions for mechanical ventilation that could improve patient outcomes.
Spatio-Temporal Vehicle Trajectory Recovery on Road Network Based on Traffic Camera Video Data
Authors: Fudan Yu, Wenxuan Ao, Huan Yan, Guozhen Zhang, Wei Wu 0021, Yong Li 0008
Venue: KDD
Year: 2022

Large-scale vehicle trajectories bring great benefits in understanding urban mobility, and can be used to promote a wide range of applications in building intelligent transportation systems. Traditional approaches cannot recover the trajectories of all the vehicles on the roads since they are based on partial trajectory data. To address it, we study the all-vehicle trajectory recovery based on traffic camera video data. However, there are two challenges in this study. First, the quality of the images captured by traffic cameras is unbalanced, so it is hard to identify the same vehicles. Second, the traffic camera observation data are sparse due to the incompleteness of the traffic cameras and possible vehicle miss from the traffic cameras. To deal with these challenges, we design a novel system to recover the vehicle trajectory with the granularity of the road intersection. In this system, we propose an iterative framework to jointly optimize the vehicle re-identification and trajectory recovery tasks. In the vehicle re-identification task, we propose an effective strategy to guide the vehicle clustering based on visual features and the spatio-temporal constraint features updated by the trajectory discovery task. In the trajectory recovery task, we model the spatial and temporal relations as well as the vehicle miss problem by a probabilistic approach to recover the trajectories. Extensive experiments demonstrate that our framework outperforms the existing state-of-art solutions. Finally, our system is deployed in practical applications of SenseTime, China, including traffic congestion analysis and traffic signal control.
CommerceMM: Large-Scale Commerce MultiModal Representation Learning with Omni Retrieval
Authors: Licheng Yu, Jun Chen, Animesh Sinha, Mengjiao Wang 0002, Yu Chen, Tamara L. Berg, Ning Zhang
Venue: KDD
Year: 2022

We introduce CommerceMM - a multimodal model capable of providing a diverse and granular understanding of commerce topics associated to the given piece of content (image, text, image+text), and having the capability to generalize to a wide range of tasks, including Multimodal Categorization, Image-Text Retrieval, Query-to-Product Retrieval, Image-to-Product Retrieval, etc. We follow the pre-training + fine-tuning training regime and present 5 effective pre-training tasks on image-text pairs. To embrace more common and diverse commerce data with text-to-multimodal, image-to-multimodal, and multimodal-to-multimodal mapping, we propose another 9 novel cross-modal and cross-pair retrieval tasks, called Omni-Retrieval pre-training. We also propose a novel approach of modality randomization to dynamically adjust our model under different efficiency constraints. The pre-training is conducted in an efficient manner with only two forward/backward updates for the combined 14 tasks. Extensive experiments and analysis show the effectiveness of each task. When combining all pre-training tasks, our model achieves state-of-the-art performance on 7 commerce-related downstream tasks after fine-tuning.
MDP2 Forest: A Constrained Continuous Multi-dimensional Policy Optimization Approach for Short-video Recommendation
Authors: Sizhe Yu, Ziyi Liu, Shixiang Wan, Jia Zheng 0005, Zang Li, Fan Zhou
Venue: KDD
Year: 2022

In the ecology of short video platforms, the optimal exposure proportion of each video category is crucial to guide recommendation systems and content production in a macroscopic way. Though extensive studies on recommendation systems are devoted to providing the most well-matched videos for each view request, fitting the data without considering inherent biases such as selection bias and exposure bias will result in serious issues. In this paper, we formalize the exposure proportion strategy as a policy-making problem with multi-dimensional continuous treatment under certain constraints from a causal inference point of view. We propose a novel ensemble policy learning method based on causal trees, called Maximum Difference of Preference Point Forest (MDP2 Forest), which overcomes the shortcomings of existing policy learning approaches. Experimental results on both simulated and synthetic datasets show the superiority of our algorithm compared to other policy learning or causal inference methods in terms of the treatment estimation accuracy and the mean regret. Furthermore, the proposed MDP2 Forest method can also adapt to a wide range of business settings such as imposing different kinds of constraints on the multi-dimensional treatment.
EGM: Enhanced Graph-based Model for Large-scale Video Advertisement Search
Authors: Tan Yu, Jie Liu, Yi Yang 0031, Yi Li, Hongliang Fei, Ping Li 0001
Venue: KDD
Year: 2022

Video advertisements may grasp customers' attention instantly and are often adored by advertisers. Since the corpus is vast, achieving an efficient query-to-video search can be challenging. Because traditional approximate nearest neighborhood (ANN) search methods are based simple similarities (e.g., cosine or inner products) on embedding vectors. They are often not sufficient for bridging the modal gap between a text query and video advertisements and typically can only achieve sub-optimal performance in query-to-video search. Tree-based deep model (TDM) overcomes the limited matching capability of embedding-based methods but suffers from the data sparsity problem. Deep retrieval model adopts a graph-based model which overcomes the data sparsity problem in TDM by sharing the nodes. But the shared nodes entangle features of different items, making it difficult to distinguish similar items. In this work, we enhance the graph-based model through sub-path embedding to differentiate similar videos. The added sub-path embedding provides personalized characteristics, beneficial for modeling fine-grain details to discriminate similar items. After launching enhanced graph model (EGM), the click-through rate (CTR) relatively increases by 1.33%, and the conversion rate (CVR) relatively by 1.07%.
PECOS: Prediction for Enormous and Correlated Output Spaces
Authors: Hsiang-Fu Yu, Jiong Zhang 0001, Wei-Cheng Chang, Jyun-Yu Jiang, Wei Li 0271, Cho-Jui Hsieh
Venue: KDD
Year: 2022

Different from traditional machine learning tasks and benchmarks, real-world problems are usually accompanied by enormous output spaces, from hundred thousands of diseases in medical diagnosis, to millions of items and billions of websites in product and web search engines. Unfortunately, conventional machine learning tools and libraries are incapable of efficiently and accurately tackling large-scale output spaces. To address this issue, PECOS (Prediction for Enormous and Correlated Output Spaces) [11] is a state-of-the-art and open-sourced machine learning library1, which not only provides high-level and user-friendly interfaces of both linear and deep learning models, but also supplies considerable flexibility for solving diverse machine learning problems. Specifically, PECOS eases complicated semantic indexing for organizing enormous output spaces, thereby efficiently training models and deriving predictions by magnitude orders on correlated output labels. As a powerful and useful framework, PECOS has already been adopted in various real- world large-scale products like semantic search in Amazon [1], as well as achieved state-of-the-art on public extreme multi-label classification (XMC) benchmarks [2, 11, 12 ] and various downstream applications [3, 7, 9].
XDAI: A Tuning-free Framework for Exploiting Pre-trained Language Models in Knowledge Grounded Dialogue Generation
Authors: Jifan Yu, Xiaohan Zhang, Yifan Xu, Xuanyu Lei, Xinyu Guan, Jing Zhang 0001, Lei Hou 0001, Juanzi Li, Jie Tang 0001
Venue: KDD
Year: 2022

Large-scale pre-trained language models (PLMs) have shown promising advances on various downstream tasks, among which dialogue is one of the most concerned. However, there remain challenges for individual developers to create a knowledge-grounded dialogue system upon such big models because of the expensive cost of collecting the knowledge resources for supporting the system as well as tuning these large models for the task. To tackle these obstacles, we propose XDAI, a knowledge-grounded dialogue system that is equipped with the prompt-aware tuning-free PLM exploitation and supported by the ready-to-use open-domain external knowledge resources plus the easy-to-change domain-specific mechanism. With XDAI, the developers can leverage the PLMs without any fine-tuning cost to quickly create the open-domain dialogue systems as well as easily customize their own domain-specific systems. Extensive experiments including human evaluation, Turing test, and online evaluation have demonstrated the competitive performance of XDAI compared with the state-of-the-art general PLMs and specific PLMs for dialogue. XDAI pilots studies on the exploitation of PLMs and made intriguing findings which could be inspiring for the future research on other PLM-based applications.
Intrinsic-Motivated Sensor Management: Exploring with Physical Surprise
Authors: Jingyi Yuan, Yang Weng, Erik Blasch
Venue: KDD
Year: 2022

In modern complex physical systems, advanced sensing technologies extend the sensor coverage but also increase the difficulties of improving system monitoring capabilities based on real-time data availability. Traditional model-based methods of sensor management are limited to specific systems/settings, which can be challenged when system knowledge is intractable. Fortunately, the large amount of data collected in real-time allows machine learning methods to be a complement. Especially, reinforcement learning-based control is recognized for its capability to dynamically interact with systems. However, the direct implementation of learning methods easily overfits and results in inaccurate physics modeling for sensor management. Although physical regularization is a popular direction to bridge the gap, learning-based sensor control still suffers from convergence failure under highly complex and uncertain scenarios. This paper develops physics-embedded and self-supervised reinforcement learning for sensor management using an intrinsic reward. Specifically, the intrinsic-motivated sensor management (IMSM) constructs the local surprise information from the physical latent features, which captures hidden states in observations, and thus intrinsically motivates the agent to speed-up exploration. We show that the designs can not only relieve the lack of consistency with underlying physics/physical dynamics, but also adapt the global objective of maximizing monitoring capabilities to local environment changes. We demonstrate its effectiveness by experiments on physical system sensor control. The proposed model is implemented for the sensor management of unmanned vehicles and sensor rescheduling in complex/settled power systems, with or without observability constraints. Numerical results show that our model provides consistently higher threat detection accuracy and better observability recovery, as compared to existing methods.
Dual Bidirectional Graph Convolutional Networks for Zero-shot Node Classification
Authors: Qin Yue 0002, Jiye Liang, Junbiao Cui, Liang Bai 0001
Venue: KDD
Year: 2022

Zero-shot node classification is a very important challenge for classical semi-supervised node classification algorithms, such as Graph Convolutional Network (GCN) which has been widely applied to node classification. In order to predict the unlabeled nodes from unseen classes, zero-shot node classification needs to transfer knowledge from seen classes to unseen classes. It is crucial to consider the relations between the classes in zero-shot node classification. However, the GCN only considers the relations between the nodes, not the relations between the classes. Therefore, the GCN can not handle the zero-shot node classification effectively. This paper proposes a Dual Bidirectional Graph Convolutional Networks (DBiGCN) that consists of dual BiGCNs from the perspective of the nodes and the classes, respectively. The BiGCN can integrate the relations between the nodes and between the classes simultaneously in an united network. In addition, to make the dual BiGCNs work collaboratively, a label consistency loss is introduced, which can achieve mutual guidance and mutual improvement between the dual BiGCNs. Finally, the experimental results on real-world graph data sets verify the effectiveness of the proposed method.
Improving Social Network Embedding via New Second-Order Continuous Graph Neural Networks
Authors: Yanfu Zhang, Shangqian Gao, Jian Pei 0001, Heng Huang 0001
Venue: KDD
Year: 2022

Graph neural networks (GNN) are powerful tools in many web research problems. However, existing GNNs are not fully suitable for many real-world web applications. For example, over-smoothing may affect personalized recommendations and the lack of an explanation for the GNN prediction hind the understanding of many business scenarios. To address these problems, in this paper, we propose a new second-order continuous GNN which naturally avoids over-smoothing and enjoys better interpretability. There is some research interest in continuous graph neural networks inspired by the recent success of neural ordinary differential equations (ODEs). However, there are some remaining problems w.r.t. the prevailing first-order continuous GNN frameworks. Firstly, augmenting node features is an essential, however heuristic step for the numerical stability of current frameworks; secondly, first-order methods characterize a diffusion process, in which the over-smoothing effect w.r.t. node representations are intrinsic; and thirdly, there are some difficulties to integrate the topology of graphs into the ODEs. Therefore, we propose a framework employing second-order graph neural networks, which usually learn a less stiff transformation than the first-order counterpart. Our method can also be viewed as a coupled first-order model, which is easy to implement. We propose a semi-model-agnostic method based on our model to enhance the prediction explanation using high-order information. We construct an analog between continuous GNNs and some famous partial differential equations and discuss some properties of the first and second-order models. Extensive experiments demonstrate the effectiveness of our proposed method, and the results outperform related baselines.
M-Mix: Generating Hard Negatives via Multi-sample Mixing for Contrastive Learning
Authors: Shaofeng Zhang, Meng Liu 0012, Junchi Yan, Hengrui Zhang, Lingxiao Huang, Xiaokang Yang 0001, Pinyan Lu
Venue: KDD
Year: 2022

Negative pairs, especially hard negatives as combined with common negatives (easy to discriminate), are essential in contrastive learning, which plays a role of avoiding degenerate solutions in the sense of constant representation across different instances. Inspired by recent hard negative mining methods via pairwise mixup operation in vision, we propose M-Mix, which dynamically generates a sequence of hard negatives. Compared with previous methods, M-Mix mainly has three features: 1) adaptively choose samples to mix; 2) simultaneously mix multiple samples; 3) automatically assign different mixing weights to the selected samples. We evaluate our method on two image datasets (CIFAR-10, CIFAR-100), five node classification datasets (PPI, DBLP, Pubmed, etc), five graph classification datasets (IMDB, PTC_MR, etc), and two downstream combinatorial tasks (graph edit distance and node clustering). Results show that it achieves state-of-the-art performance under self-supervised settings. Code is available at: https://github.com/Sherrylone/m-mix.
CAT: Beyond Efficient Transformer for Content-Aware Anomaly Detection in Event Sequences
Authors: Shengming Zhang, Yanchi Liu, Xuchao Zhang, Wei Cheng 0002, Haifeng Chen, Hui Xiong 0001
Venue: KDD
Year: 2022

It is critical and important to detect anomalies in event sequences, which becomes widely available in many application domains. Indeed, various efforts have been made to capture abnormal patterns from event sequences through sequential pattern analysis or event representation learning. However, existing approaches usually ignore the semantic information of event content. To this end, in this paper, we propose a self-attentive encoder-decoder transformer framework, Content-Aware Transformer CAT, for anomaly detection in event sequences. In CAT, the encoder learns preamble event sequence representations with content awareness, and the decoder embeds sequences under detection into a latent space, where anomalies are distinguishable. Specifically, the event content is first fed to a content-awareness layer, generating representations of each event. The encoder accepts preamble event representation sequence, generating feature maps. In the decoder, an additional token is added at the beginning of the sequence under detection, denoting the sequence status. A one-class objective together with sequence reconstruction loss is collectively applied to train our framework under the label efficiency scheme. Furthermore, CAT is optimized under a scalable and efficient setting. Finally, extensive experiments on three real-world datasets demonstrate the superiority of CAT.