OpenRaiser/Intern-Atlas huggingface dataset:
A Counterfactual Modeling Framework for Churn Prediction
Authors: Guozhen Zhang, Jinwei Zeng, Zhengyue Zhao, Depeng Jin, Yong Li
Venue: WSDM
Year: 2022

Accurate churn prediction for retaining users is keenly important for online services because it determines their survival and prosperity. Recent research has specified social influence to be one of the most important reasons for user churn, and thereby many works start to model its effects on user churn to improve the prediction performance. However, existing works only use the data's correlational information while neglecting the problem's causal nature. Specifically, the fact that a user's churn is correlated with some social factors does not mean he/she is actually influenced by his/her friends, which results in inaccurate and unexplainable predictions of the existing methods. To bridge this gap, we develop a counterfactual modeling framework for churn prediction, which can effectively capture the causal information of social influence for accurate and explainable churn predictions. Specifically, we first propose a backbone framework that uses two separate embeddings to model users' endogenous churn intentions and the exogenous social influence. Then, we propose a counterfactual data augmentation module to introduce the causal information to the model by providing partially labeled counterfactual data. Finally, we design a three-headed counterfactual prediction framework to guide the model to learn causal information to facilitate churn prediction. Extensive experiments on two large-scale datasets with different types of social relations show our model's superior prediction performance compared with the state-of-the-art baselines. We further conduct an in-depth analysis of the prediction results demonstrating our proposed method's ability to capture causal information of social influence and give explainable churn predictions, which provide insights into designing better user retention strategies.
ST-GSP: Spatial-Temporal Global Semantic Representation Learning for Urban Flow Prediction
Authors: Liang Zhao, Min Gao, Zongwei Wang
Venue: WSDM
Year: 2022

Urban flow prediction plays a crucial role in public transportation management and smart city construction. Although previous studies have achieved success in integrating spatial-temporal information to some extents, those models lack thoughtful consideration on global information and positional information in the temporal dimension, which can be summarized by three aspects: a) The models do not consider the relative position information of time axis, resulting in that the position features of flow maps are not effectively learned. b) They overlook the correlation among temporal dependencies of different scales, which lead to inaccurate global information representation. c) Those models only predict the flow map at the end of time sequence other than more flow maps before that, which results in neglecting parts of temporal features in the learning process. To solve the problems, we propose a novel model, Spatial-Temporal Global Semantic representation learning for urban flow Prediction (ST-GSP) in this paper. Specifically, for a), we design a semantic flow encoder that extracts relative positional information of time. Besides, the encoder captures the spatial dependencies and external factors of urban flow at each time interval. For b), we model the correlation among temporal dependencies of different scales simultaneously by using the multi-head self-attention mechanism, which can learn the global temporal dependencies. For c), inspired by the idea of self-supervised learning, we mask an urban flow map on the time sequence and predict it to pre-train a deep bidirectional learning model to catch the representation from its context. We conduct extensive experiments on two types of urban flows in Beijing and New York City to show that the proposed method outperforms state-of-the-art methods.
Multi-Sparse-Domain Collaborative Recommendation via Enhanced Comprehensive Aspect Preference Learning
Authors: Xiaoyun Zhao, Ning Yang, Philip S. Yu
Venue: WSDM
Year: 2022

Cross-domain recommendation (CDR) has been attracting increasing attention of researchers for its ability to alleviate the data sparsity problem in recommender systems. However, the existing single-target or dual-target CDR methods often suffer from two drawbacks, the assumption of at least one rich domain and the heavy dependence on domain-invariant preference, which are impractical in real world where sparsity is ubiquitous and might degrade the user preference learning. To overcome these issues, we propose a Multi-Sparse-Domain Collaborative Recommendation (MSDCR) model for multi-target cross-domain recommendation. Unlike traditional CDR methods, MSDCR treats the multiple relevant domains as all sparse and can simultaneously improve the recommendation performance in each domain. We propose a Multi-Domain Separation Network (MDSN) and a Gated Aspect Preference Enhancement (GAPE) module for MSDCR to enhance a user's domain-specific aspect preferences in a domain by transferring the complementary aspect preferences in other domains, during which the uniqueness of the domain-specific preference can be preserved through the adversarial training offered by MDSN and the complementarity can be adaptively determined by GAPE. Meanwhile, we propose a Multi-Domain Adaptation Network (MDAN) for MSDCR to capture a user's domain-invariant aspect preference. With the integration of the enhanced domain-specific aspect preference and the domain-invariant aspect preference, MSDCR can reach a comprehensive understanding of a user's preference in each sparse domain. At last, the extensive experiments conducted on real datasets demonstrate the remarkable superiority of MSDCR over the state-of-the-art single-domain recommendation models and CDR models.
Joint Learning of E-commerce Search and Recommendation with a Unified Graph Neural Network
Authors: Kai Zhao, Yukun Zheng, Tao Zhuang, Xiang Li, Xiaoyi Zeng
Venue: WSDM
Year: 2022

Click-through rate (CTR) prediction plays an important role in search and recommendation, which are the two most prominent scenarios in e-commerce. A number of models have been proposed to predict CTR by mining user behaviors, especially users' interactions with items. But the sparseness of user behaviors is an obstacle to the improvement of CTR prediction. Previous works only focused on one scenario, either search or recommendation. However, on a practical e-commerce platform, search and recommendation share the same set of users and items, which means joint learning of both scenarios may alleviate the sparseness of user behaviors. In this paper, we propose a novel Search and Recommendation Joint Graph (SRJGraph) neural network to jointly learn a better CTR model for both scenarios. A key question of joint learning is how to effectively share information across search and recommendation, in spite of their differences. A notable difference between search and recommendation is that there are explicit queries in search, whereas no query exists in recommendation. We address this difference by constructing a unified graph to share representations of users and items across search and recommendation, as well as represent user-item interactions uniformly. In this graph, users and items are heterogeneous nodes, and search queries are incorporated into the user-item interaction edges as attributes. For recommendation where no query exists, a special attribute is attached on user-item interaction edges. We further propose an intention and upstream-aware aggregator to explore useful information from high-order connections among users and items. We conduct extensive experiments on a large-scale dataset collected from Taobao.com, the largest e-commerce platform in China. Empirical results show that SRJGraph significantly outperforms the state-of-the-art approaches of CTR prediction in both search and recommendation tasks.
Learning Transferable Node Representations for Attribute Extraction from Web Documents
Authors: Yichao Zhou, Ying Sheng, Nguyen Vo, Nick Edmonds, Sandeep Tata
Venue: WSDM
Year: 2022

Given a web page, extracting an object along with various attributes of interest (e.g. price, publisher, author, and genre for a book) can facilitate a variety of downstream applications such as large-scale knowledge base construction, e-commerce product search, and personalized recommendation. Prior approaches have either relied on computationally expensive visual feature engineering or required large amounts of training data to get to an acceptable precision. In this paper, we propose a novel method, LeArNing TransfErable node RepresentatioNs for Attribute Extraction (LANTERN), to tackle the problem. We model the problem as a tree node tagging task. The key insight is to learn a contextual representation for each node in the DOM tree where the context explicitly takes into account the tree structure of the neighborhood around the node. Experiments on the SWDE public dataset show that LANTERN outperforms the previous state-of-the-art (SOTA) by 1.44% (F1 score) with a dramatically simpler model architecture. Furthermore, we report that utilizing data from a different domain (for instance, using training data about web pages with cars to extract book objects) is surprisingly useful and helps beat the SOTA by a further 1.37%.
AngHNE: Representation Learning for Bipartite Heterogeneous Networks with Angular Loss
Authors: Cangqi Zhou, Hui Chen, Jing Zhang, Qianmu Li, Dianming Hu
Venue: WSDM
Year: 2022

Real-world networks often show heterogeneity. A frequently encountered type is the bipartite heterogeneous structure, in which two types of nodes and three types of edges exist. Recently, much attention has been devoted to representation learning in these networks. One of the essential differences between heterogeneous and homogeneous learning is that the former structure requires methods to possess awareness to node and edge types. Most existing methods, including metapath-based, proximity-based and graph neural network-based, adopt inner product or vector norms to evaluate the similarities in embedding space. However, these measures either violates the triangle inequality, or show severe sensitivity to scaling transformation. The limitations often hinder the applicability to real-world problems. In view of this, in this paper, we propose a novel angle-based method for bipartite heterogeneous network representation. Specifically, we first construct training sets by generating quintuples, which contain both positive and negative samples from two different parts of networks. Then we analyze the quintuple-based problem from a geometry perspective, and transform the comparisons between preferred and non-preferred samples to the comparisons of angles. In addition, we utilize convolution modules to extract node features. A hinge loss, as the final objective, is proposed to relax the angular constraint for learning. Extensive experiments for two typical tasks show the efficacy of the proposed method, comparing with eight competitive methods.
C²-CRS: Coarse-to-Fine Contrastive Learning for Conversational Recommender System
Authors: Yuanhang Zhou, Kun Zhou, Wayne Xin Zhao, Cheng Wang, Peng Jiang, He Hu
Venue: WSDM
Year: 2022

Conversational recommender systems (CRS) aim to recommend suitable items to users through natural language conversations. For developing effective CRSs, a major technical issue is how to accurately infer user preference from very limited conversation context. To address issue, a promising solution is to incorporate external data for enriching the context information. However, prior studies mainly focus on designing fusion models tailored for some specific type of external data, which is not general to model and utilize multi-type external data. To effectively leverage multi-type external data, we propose a novel coarse-to-fine contrastive learning framework to improve data semantic fusion for CRS. In our approach, we first extract and represent multi-grained semantic units from different data signals, and then align the associated multi-type semantic units in a coarse-to-fine way. To implement this framework, we design both coarse-grained and fine-grained procedures for modeling user preference, where the former focuses on more general, coarse-grained semantic fusion and the latter focuses on more specific, fine-grained semantic fusion. Such an approach can be extended to incorporate more kinds of external data. Extensive experiments on two public CRS datasets have demonstrated the effectiveness of our approach in both recommendation and conversation tasks.
Fighting Mainstream Bias in Recommender Systems via Local Fine Tuning
Authors: Ziwei Zhu, James Caverlee
Venue: WSDM
Year: 2022

In collaborative filtering, the quality of recommendations critically relies on how easily a model can find similar users for a target user. Hence, a niche user who prefers items out of the mainstream may receive poor recommendations, while a mainstream user sharing interests with many others will likely receive recommendations of higher quality. In this work, we study this mainstream bias centering around three key thrusts. First, to distinguish mainstream and niche users, we explore four approaches based on outlier detection techniques to identify a mainstream score indicating the mainstream level for each user. Second, we empirically show that severe mainstream bias is produced by conventional recommendation models. Last, we explore both global and local methods to mitigate the bias. Concretely, we propose two global models: Distribution Calibration (DC) and Weighted Loss (WL) methods; and one local method: Local Fine Tuning (LFT) method. Extensive experiments show the effectiveness of the proposed methods to improve utility for niche users and also show that the proposed LFT can improve the utility for mainstream users at the same time.
A Neighborhood-Attention Fine-grained Entity Typing for Knowledge Graph Completion
Authors: Jianhuan Zhuo, Qiannan Zhu, Yinliang Yue, Yuhong Zhao, Weisi Han
Venue: WSDM
Year: 2022

Knowledge graph (KG) entity typing focuses on inferring possible entity type instances, which is a significant subtask of knowledge graph completion (KGC). Existing entity typing methods usually exploit the entity representation to model the transmission between entities and their types, which cannot fully explore the fine-grained entity typing on identifying the semantic type of an entity. To address these issues, we propose Neighborhood-Attention Neural Fine-Grained Entity Typing (AttEt), which considers the neighborhood information of the entities from KGs to bridge entities and their types together. In this paper, AttEt first develops a type-specific attention mechanism to aggregate the neighborhood knowledge of the given entity with type-specific weights. These weights are beneficial to capture various characteristics for different types of the entity, and further imply the complex correlation among these fine-grained types. Then, AttEt adaptively integrates the aggregated neighbor-level representation with entity inherent embedding to calculate the matching score between the entity and its candidate type. Besides, many entities are sparse in their relations with other entities in KGs, which makes the entity typing task more challenging. To solve this problem, we present a smooth strategy on relation-sparsity entities to improve the robustness of the model. Extensive experiments on two real-world datasets (Freebase and YAGO) show that AttEt significantly outperforms state-of-the-art baselines in the [email protected] by 2.11% on Freebase and by 8.42% on YAGO, respectively.
Improving Session Search by Modeling Multi-Granularity Historical Query Change
Authors: Xiaochen Zuo, Zhicheng Dou, Ji-Rong Wen
Venue: WSDM
Year: 2022

In session search, it's important to utilize historical interactions between users and the search engines to improve document retrieval. However, not all historical information contributes to document ranking. Users often express their preferences in the process of modifying the previous query, which can help us catch useful information in the historical interactions. Inspired by it, we propose to model historical query change to improve document ranking performance. Especially, we characterize multi-granularity query change between each pair of adjacent queries at both term level and semantic level. For term level query change, we calculate three types of term weights, including the retained term weights, added term weights and removed term weights. Then we perform term-based interaction between the candidate document and historical queries based on the term weights. For semantic level query change, we calculate an overall representation of user intent by integrating the representations of each historical query obtained by different types of term weights. Then we adopt representation-based matching between this representation and the candidate document. To improve the effect of query change modeling, we introduce query change classification as an auxiliary task. Experimental results on AOL and TianGong-ST search logs show that our model outperforms most existing models for session search.
Recent Advances on Deep Learning based Knowledge Tracing
Authors: Zitao Liu, Jiahao Chen, Weiqi Luo
Venue: WSDM
Year: 2023

Knowledge tracing (KT) is the task of using students' historical learning interaction data to model their knowledge mastery over time so as to make predictions on their future interaction performance. Recently, remarkable progress has been made of using various deep learning techniques to solve the KT problem. However, the success behind deep learning based knowledge tracing (DLKT) approaches is still left somewhat unknown and proper measurement and analysis of these DLKT approaches remain a challenge. In this talk, we will comprehensively review recent developments of applying state-of-the-art deep learning approaches in KT problems, with a focus on those real-world educational data. Beyond introducing the recent advances of various DLKT models, we will discuss how to guarantee valid comparisons across DLKT methods via thorough evaluations on several publicly available datasets. More specifically, we will talk about (1) KT related psychometric theories; (2) the general DLKT modeling framework that covers recently developed DLKT approaches from different categories; (3) the general DLKT benchmark that allows existing approaches comparable on public KT datasets; (4) the broad application of algorithmic assessment and personalized feedback. Participants will learn about recent trends and emerging challenges in this topic, representative tools and learning resources to obtain ready-to-use models, and how related models and techniques benefit real-world KT applications.
Incorporating Fairness in Large Scale NLU Systems
Authors: Rahul Gupta, Lisa Bauer, Kai-Wei Chang, Jwala Dhamala, Aram Galstyan, Palash Goyal, Qian Hu, Avni Khatri, Rohit Parimi, Charith Peris, Apurv Verma, Richard S. Zemel, Prem Natarajan
Venue: WSDM
Year: 2023

NLU models power several user facing experiences such as conversations agents and chat bots. Building NLU models typically consist of 3 stages: a) building or finetuning a pre-trained model b) distilling or fine-tuning the pre-trained model to build task specific models and, c) deploying the task-specific model to production. In this presentation, we will identify fairness considerations that can be incorporated in the aforementioned three stages in the life-cycle of NLU model building: (i) selection/building of a large scale language model, (ii) distillation/fine-tuning the large model into task specific model and, (iii) deployment of the task specific model. We will present select metrics that can be used to quantify fairness in NLU models and fairness enhancement techniques that can be deployed in each of these stages. Finally, we will share some recommendations to successfully implement fairness considerations when building an industrial scale NLU system.
DeMEtRIS: Counting (near)-Cliques by Crawling
Authors: Suman K. Bera, Jayesh Choudhari, Shahrzad Haddadan, Sara Ahmadian
Venue: WSDM
Year: 2023

We study the problem of approximately counting cliques and near cliques in a graph, where the access to the graph is only available through crawling its vertices; thus typically seeing only a small portion of it. This model, known as the random walk model or the neighborhood query model has been introduced recently and captures real-life scenarios in which the entire graph is too massive to be stored as a whole or be scanned entirely and sampling vertices independently is non-trivial in it. We introduce DeMEtRIS: Dense Motif Estimation through Random Incident Sampling. This method provides a scalable algorithm for clique and near clique counting in the random walk model. We prove the correctness of our algorithm through rigorous mathematical analysis and extensive experiments. Both our theoretical results and our experiments show that DeMEtRIS obtains a high precision estimation by only crawling a sub-linear portion on vertices, thus we demonstrate a significant improvement over previously known results.
Towards an Event-Aware Urban Mobility Prediction System
Authors: Zhaonan Wang, Renhe Jiang, Zipei Fan, Xuan Song, Ryosuke Shibasaki
Venue: WSDM
Year: 2023

Today, thanks to the rapid developing mobile and sensor networks in IoT (Internet of Things) systems, spatio-temporal big data are being constantly generated. They have brought us a data-driven possibility to sense and understand crowd mobility on a city scale. A fundamental task towards the next-generation mobility services, such as Intelligent Transportation Systems (ITS), Mobility-as-a-Service (MaaS), is spatio-temporal predictive modeling of the geo-sensory signals. There is a recent line of research leveraging deep learning techniques to boost the forecasting performance on such tasks. While simulating the regularity of mobility behaviors (e.g., routines, periodicity) in a more sophisticated way, the existing studies ignore an important part of urban activities, i.e., events. Including holidays, extreme weathers, pandemic, accidents, various urban events happen from time to time and cause non-stationary phenomena, which by nature make the spatio-temporal forecasting task challenging. We thereby envision an event-aware urban mobility prediction model that is capable of fast adapting and making reliable predictions in different scenarios, which is crucial to decision making towards emergency response and urban resilience.
A Tutorial on Domain Generalization
Authors: Jindong Wang, Haoliang Li, Sinno Jialin Pan, Xing Xie
Venue: WSDM
Year: 2023

With the availability of massive labeled training data, powerful machine learning models can be trained. However, the traditional I.I.D. assumption that the training and testing data should follow the same distribution is often violated in reality. While existing domain adaptation approaches can tackle domain shift, it relies on the target samples for training. Domain generalization is a promising technology that aims to train models with good generalization ability to unseen distributions. In this tutorial, we will present the recent advance of domain generalization. Specifically, we introduce the background, formulation, and theory behind this topic. Our primary focus is on the methodology, evaluation, and applications. We hope this tutorial can draw interest of the community and provide a thorough review of this area. Eventually, more robust systems can be built for responsible AI. All tutorial materials and updates can be found online at https://dgresearch.github.io/.