OpenRaiser/Intern-Atlas huggingface dataset:
Ada-GNN: Adapting to Local Patterns for Improving Graph Neural Networks
Authors: Zihan Luo, Jianxun Lian, Hong Huang, Hai Jin, Xing Xie
Venue: WSDM
Year: 2022

Graph Neural Networks (GNNs) have demonstrated strong power in mining various graph-structure data. Since real-world graphs are usually on a large scale, training scalable GNNs has become one of the research trends in recent years. Existing methods only produce one single model to serve all nodes. However, different nodes may exhibit various properties thus require diverse models, especially when the graph is large. Forcing all nodes to share a unified model will decrease the model's expressiveness. What is worse, some small groups' patterns are prone to be ignored by the model due to their minority, making these nodes unpredictable and even some raising potential unfairness problems. In this paper, we propose a model-agnostic framework Ada-GNN that provides personalized GNN models for specific sets of nodes. Intuitively, it is desirable that every node has its own model. But considering the efficiency and scalability of the framework, we generate specific GNN models at the subgroup-level rather than individual node-level. To be specific, Ada-GNN first splits the original graph into several non-overlapped subgroups and tags each node with its subgroup label. After that, a meta adapter is proposed to adapt a base GNN model to each subgroup rapidly. To better facilitate the global-to-local knowledge adaption, we design a feature enhancement module that captures the distinctions among different subgroups to improve the Ada-GNN's performance. Ada-GNN is model-agnostic and can be equipped to almost all existing scalable GNN based methods such as GraphSAGE, ClusterGCN, SIGN, and SAGN. We conduct extensive experiments with six popular scalable GNN as base methods on two large-scale datasets, and the results consistently demonstrate the generality and superiority of Ada-GNN.
RLMob: Deep Reinforcement Learning for Successive Mobility Prediction
Authors: Ziyan Luo, Congcong Miao
Venue: WSDM
Year: 2022

Human mobility prediction is an important task in the field of spatiotemporal sequential data mining and urban computing. Despite the extensive work on mining human mobility behavior, little attention was paid to the problem of successive mobility prediction. The state-of-the-art methods of human mobility prediction are mainly based on supervised learning. To achieve higher predictability and adapt well to the successive mobility prediction, there are four key challenges: 1) disability to the circumstance that the optimizing target is discrete-continuous hybrid and non-differentiable. In our work, we assume that the user's demands are always multi-targeted and can be modeled as a discrete-continuous hybrid function; 2) difficulty to alter the recommendation strategy flexibly according to the changes in user needs in real scenarios; 3) error propagation and exposure bias issues when predicting multiple points in successive mobility prediction; 4) cannot interactively explore user's potential interest that does not appear in the history. While previous methods met these difficulties, reinforcement learning (RL) is an intuitive answer for this task to settle these issues. We innovatively introduce RL to the successive prediction task. In this paper, we formulate this problem as a Markov Decision Process. We further propose a framework - RLMob to solve our problem. A simulated environment is carefully designed. An actor-critic framework with an instance of Proximal Policy Optimization (PPO) is applied to adapt to our scene with a large state space. Experiments show that on the task, the performance of our approach is consistently superior to that of the compared approaches.
Learning Fair Node Representations with Graph Counterfactual Fairness
Authors: Jing Ma, Ruocheng Guo, Mengting Wan, Longqi Yang, Aidong Zhang, Jundong Li
Venue: WSDM
Year: 2022

Fair machine learning aims to mitigate the biases of model predictions against certain subpopulations regarding sensitive attributes such as race and gender. Among the many existing fairness notions, counterfactual fairness measures the model fairness from a causal perspective by comparing the predictions of each individual from the original data and the counterfactuals. In counterfactuals, the sensitive attribute values of this individual had been modified. Recently, a few works extend counterfactual fairness to graph data, but most of them neglect the following facts that can lead to biases: 1) the sensitive attributes of each node's neighbors may causally affect the prediction w.r.t. this node; 2) the sensitive attributes may causally affect other features and the graph structure. To tackle these issues, in this paper, we propose a novel fairness notion - graph counterfactual fairness, which considers the biases led by the above facts. To learn node representations towards graph counterfactual fairness, we propose a novel framework based on counterfactual data augmentation. In this framework, we generate counterfactuals corresponding to perturbations on each node's and their neighbors' sensitive attributes. Then we enforce fairness by minimizing the discrepancy between the representations learned from the original graph and the counterfactuals for each node. Experiments on both synthetic and real-world graphs show that our framework outperforms the state-of-the-art baselines in graph counterfactual fairness, and also achieves comparable prediction performance.
Deep Graph-level Anomaly Detection by Glocal Knowledge Distillation
Authors: Rongrong Ma, Guansong Pang, Ling Chen, Anton van den Hengel
Venue: WSDM
Year: 2022

Graph-level anomaly detection (GAD) describes the problem of detecting graphs that are abnormal in their structure and/or the features of their nodes, as compared to other graphs. One of the challenges in GAD is to devise graph representations that enable the detection of both locally- and globally-anomalous graphs, i.e., graphs that are abnormal in their fine-grained (node-level) or holistic (graph-level) properties, respectively. To tackle this challenge we introduce a novel deep anomaly detection approach for GAD that learns rich global and local normal pattern information by joint random distillation of graph and node representations. The random distillation is achieved by training one GNN to predict another GNN with randomly initialized network weights. Extensive experiments on 16 real-world graph datasets from diverse domains show that our model significantly outperforms seven state-of-the-art models. Code and datasets are available at https://git.io/GLocalKD.
Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations
Authors: Ganeshan Malhotra, Abdul Waheed, Aseem Srivastava, Md. Shad Akhtar, Tanmoy Chakraborty
Venue: WSDM
Year: 2022

The onset of the COVID-19 pandemic has brought the mental health of people under risk. Social counselling has gained remarkable significance in this environment. Unlike general goal-oriented dialogues, a conversation between a patient and a therapist is considerably implicit, though the objective of the conversation is quite apparent. In such a case, understanding the intent of the patient is imperative in providing effective counselling in therapy sessions, and the same applies to a dialogue system as well. In this work, we take forward a small but an important step in the development of an automated dialogue system for mental-health counselling. We develop a novel dataset, named HOPE, to provide a platform for the dialogue-act classification in counselling conversations. We identify the requirement of such conversation and propose twelve domain-specific dialogue-act (DAC) labels. We collect ~ 12.9K utterances from publicly-available counselling session videos on YouTube, extract their transcripts, clean, and annotate them with DAC labels. Further, we propose SPARTA, a transformer-based architecture with a novel speaker- and time-aware contextual learning for the dialogue-act classification. Our evaluation shows convincing performance over several baselines, achieving state-of-the-art on HOPE. We also supplement our experiments with extensive empirical and qualitative analyses of SPARTA.
The Rise of Data Observability: Architecting the Future of Data Trust
Authors: Barr Moses
Venue: WSDM
Year: 2022

As companies become increasingly data driven, the technologies underlying these rich insights have grown more and more nuanced and complex. While our ability to collect, store, aggregate, and visualize this data has largely kept up with the needs of modern data teams (think: domain-oriented data meshes, cloud warehouses, data visualization tools, and data modeling solutions), the mechanics behind data quality and integrity has lagged. To keep pace with data's clock speed of innovation, data engineers need to invest not only in the latest modeling and analytics tools, but also technologies that can increase data accuracy and prevent broken pipelines. The solution? Data observability, the next frontier of data engineering. I'll discuss why data observability matters to building a better data quality strategy and tactics best-in-class organizations use to address it -- including org structure, culture, and technology.
Mining Frequent Patterns on Knowledge Graphs
Authors: Lalla Mouatadid
Venue: WSDM
Year: 2022

The adoption of knowledge graphs is growing across various domains. And their construction, when automated, can result in verbose sparse graphs. In this talk, we discuss how to mine frequent patterns and subgraphs in a calculation knowledge graph in order to factor out verbose representations and allow for each of maintenance, reduction of storage, and an increase in runtime speed.
Successes and Opportunities in Enterprise AI
Authors: Vijay K. Narayanan
Venue: WSDM
Year: 2022

The advances in AI over the last decade has led to significantly better outcomes and improved experiences in consumer applications. Meanwhile, successful applications of AI in enterprises have been modest during this period. In this talk, I will provide an overview of the scope of Enterprise AI, how it is similar and different from AI for consumer applications, a few successful applications, open problems and challenges and the enormous opportunity to create better outcomes and transform the experiences for employees and customers of enterprises.
Node Co-occurrence based Graph Neural Networks for Knowledge Graph Link Prediction
Authors: Dai Quoc Nguyen, Vinh Tong, Dinh Q. Phung, Dat Quoc Nguyen
Venue: WSDM
Year: 2022

We introduce a novel embedding model, named NoGE, which aims to integrate co-occurrence among entities and relations into graph neural networks to improve knowledge graph completion (i.e., link prediction). Given a knowledge graph, NoGE constructs a single graph considering entities and relations as individual nodes. NoGE then computes weights for edges among nodes based on the co-occurrence of entities and relations. Next, NoGE proposes Dual Quaternion Graph Neural Networks (DualQGNN) and utilizes DualQGNN to update vector representations for entity and relation nodes. NoGE then adopts a score function to produce the triple scores. Comprehensive experimental results show that NoGE obtains state-of-the-art results on three new and difficult benchmark datasets CoDEx for knowledge graph completion.
Reconfiguration Problems on Submodular Functions
Authors: Naoto Ohsaka, Tatsuya Matsuoka
Venue: WSDM
Year: 2022

\emphReconfiguration problems require finding a step-by-step transformation between a pair of feasible solutions for a particular problem. The primary concern in Theoretical Computer Science has been revealing their computational complexity for classical problems. This paper presents an initial study on reconfiguration problems derived from a submodular function, which has more of a flavor of Data Mining. Our submodular reconfiguration problems request to find a solution sequence connecting two input solutions such that each solution has an objective value above a threshold in a submodular function $f: 2^[n] \to \mathbbR _+$ and is obtained from the previous one by applying a simple transformation rule. We formulate three reconfiguration problems: Monotone Submodular Reconfiguration (MSReco), which applies to influence maximization, and two versions of Unconstrained Submodular Reconfiguration (USReco), which apply to determinantal point processes. Our contributions are summarized as follows: \beginitemize \item We prove that MSReco and USReco are both PSPACE-complete. \item We design a $\frac1 2 $-approximation algorithm for MSReco and a $\frac1 n $-approximation algorithm for (one version of) USReco. \item We devise inapproximability results that approximating the optimum value of MSReco within a $(1-\frac1+ε n^2 )$-factor is PSPACE-hard, and we cannot find a $(\frac5 6 +ε)$-approximation for USReco. \item We conduct numerical study on the reconfiguration version of influence maximization and determinantal point processes using real-world social network and movie rating data. \enditemize
Heterogeneous Global Graph Neural Networks for Personalized Session-based Recommendation
Authors: Yitong Pang, Lingfei Wu, Qi Shen, Yiming Zhang, Zhihua Wei, Fangli Xu, Ethan Chang, Bo Long, Jian Pei
Venue: WSDM
Year: 2022

Predicting the next interaction of a short-term interaction session is a challenging task in session-based recommendation. Almost all existing works rely on item transition patterns, and neglect user historical sessions while modeling user preference, which often leads to non-personalized recommendation. And existing personalized session-based recommenders are limited to sessions of the current user, and ignore the useful item-transition patterns from other user's historical sessions. To address these issues, we propose a novel Heterogeneous Global Graph Neural Networks (HG-GNN) to exploit the item transitions over all sessions in a subtle manner for better inferring user preference from the current and historical sessions. To effectively exploit the item transitions over all sessions from users, our global graph contains item transitions of sessions, user-item interactions and global co-occurrence items. Moreover, to capture user preference from sessions comprehensively, we propose a graph augmented preference encoder to learn the session representation. Specifically, we design a novel heterogeneous graph neural network (HGNN) on heterogeneous global graph to learn long-term user preference and item representations with rich semantics. Based on the HGNN, we propose the Personalized Session Encoder to combine the general user preference and temporal interest of the current session to generate the personalized session representation for recommendation. Extensive experimental results on three real-world datasets show that our model outperforms other state-of-the-art methods.
Reinforcement Learning over Sentiment-Augmented Knowledge Graphs towards Accurate and Explainable Recommendation
Authors: Sung-Jun Park, Dong-Kyu Chae, Hong-Kyun Bae, Sumin Park, Sang-Wook Kim
Venue: WSDM
Year: 2022

Explainable recommendation has gained great attention in recent years. A lot of work in this research line has chosen to use the knowledge graphs (KG) where relations between entities can serve as explanations. However, existing studies have not considered sentiment on relations in KG, although there can be various types of sentiment on relations worth considering (e.g., a user's satisfaction on an item). In this paper, we propose a novel recommendation framework based on KG integrated with sentiment analysis for more accurate recommendation as well as more convincing explanations. To this end, we first construct a Sentiment-Aware Knowledge Graph (namely, SAKG) by analyzing reviews and ratings on items given by users. Then, we perform item recommendation and reasoning over SAKG through our proposed Sentiment-Aware Policy Learning (namely, SAPL) based on a reinforcement learning strategy. To enhance the explainability for end-users, we further developed an interactive user interface presenting textual explanations as well as a collection of reviews related with the discovered sentiment. Experimental results on three real-world datasets verified clear improvements on both the accuracy of recommendation and the quality of explanations.
EvoKG: Jointly Modeling Event Time and Network Structure for Reasoning over Temporal Knowledge Graphs
Authors: Namyong Park, Fuchen Liu, Purvanshi Mehta, Dana Cristofor, Christos Faloutsos, Yuxiao Dong
Venue: WSDM
Year: 2022

How can we perform knowledge reasoning over temporal knowledge graphs (TKGs)? TKGs represent facts about entities and their relations, where each fact is associated with a timestamp. Reasoning over TKGs, i.e., inferring new facts from time-evolving KGs, is crucial for many applications to provide intelligent services. However, despite the prevalence of real-world data that can be represented as TKGs, most methods focus on reasoning over static knowledge graphs, or cannot predict future events. In this paper, we present a problem formulation that unifies the two major problems that need to be addressed for an effective reasoning over TKGs, namely, modeling the event time and the evolving network structure. Our proposed method EvoKG jointly models both tasks in an effective framework, which captures the ever-changing structural and temporal dynamics in TKGs via recurrent event modeling, and models the interactions between entities based on the temporal neighborhood aggregation framework. Further, EvoKG achieves an accurate modeling of event time, using flexible and efficient mechanisms based on neural density estimation. Experiments show that EvoKG outperforms existing methods in terms of effectiveness (up to 77% and 116% more accurate time and link prediction) and efficiency.
Experiments with Predictive Long Term Guardrail Metrics
Authors: Sri Sri Perangur
Venue: WSDM
Year: 2022

Product experiments today need a long term view of impact to make shipping decisions truly effective. Here we will discuss the challenges in the traditional metrics used in experiment analysis and how long term forecast metrics enable better decisions. Most tech companies such as Google, Amazon, Netflix etc run thousands of experiments (also known as A/B test) a year [1]. The aim is to measure the impact new features have on core Key Predictive Indicators (KPIs) before deciding to launch it to production. Traditional A/B testing metrics will usually measure the impact of the feature on core KPIs in the short-term. However, for many lines of business (such as loyalty and memberships), this is not enough, as we want to understand the impact of the features in the mid/long term. This reality can force companies to run experiments to 6+ months duration, or use a correlated leading metric (such as user activity, engagement level) with estimated impact in the long term. Both these situations are not ideal, the first slows down the rate of innovation while the second does not account for multiple factors that define the future results. At Lyft, this reality is shared, and one that becomes a challenge for innovation as we need to know the long term impact before we decide to ship new features. As a solution we design forecasted metrics for retention and revenue at a user level that can be used to measure the impact of experiments in the long term. In this talk we will discuss challenges and learnings from this approach, when applied in practice.
Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation
Authors: Ruihong Qiu, Zi Huang, Hongzhi Yin, Zijian Wang
Venue: WSDM
Year: 2022

Recent advancements of sequential deep learning models such as Transformer and BERT have significantly facilitated the sequential recommendation. However, according to our study, the distribution of item embeddings generated by these models tends to degenerate into an anisotropic shape, which may result in high semantic similarities among embeddings. In this paper, both empirical and theoretical investigations of this representation degeneration problem are first provided, based on which a novel recommender model DuoRec is proposed to improve the item embeddings distribution. Specifically, in light of the uniformity property of contrastive learning, a contrastive regularization is designed for DuoRec to reshape the distribution of sequence representations. Given the convention that the recommendation task is performed by measuring the similarity between sequence representations and item embeddings in the same space via dot product, the regularization can be implicitly applied to the item embedding distribution. Existing contrastive learning methods mainly rely on data level augmentation for user-item interaction sequences through item cropping, masking, or reordering and can hardly provide semantically consistent augmentation samples. In DuoRec, a model-level augmentation is proposed based on Dropout to enable better semantic preserving. Furthermore, a novel sampling strategy is developed, where sequences having the same target item are chosen hard positive samples. Extensive experiments conducted on five datasets demonstrate the superior performance of the proposed DuoRec model compared with baseline methods. Visualization results of the learned representations validate that DuoRec can largely alleviate the representation degeneration problem.
Time Masking for Temporal Language Models
Authors: Guy D. Rosin, Ido Guy, Kira Radinsky
Venue: WSDM
Year: 2022

Our world is constantly evolving, and so is the content on the web. Consequently, our languages, often said to mirror the world, are dynamic in nature. However, most current contextual language models are static and cannot adapt to changes over time. In this work, we propose a temporal contextual language model called TempoBERT, which uses time as an additional context of texts. Our technique is based on modifying texts with temporal information and performing time masking - specific masking for the supplementary time information. We leverage our approach for the tasks of semantic change detection and sentence time prediction, experimenting on diverse datasets in terms of time, size, genre, and language. Our extensive evaluation shows that both tasks benefit from exploiting time masking.
On Sampling Collaborative Filtering Datasets
Authors: Noveen Sachdeva, Carole-Jean Wu, Julian J. McAuley
Venue: WSDM
Year: 2022

We study the practical consequences of dataset sampling strategies on the ranking performance of recommendation algorithms. Recommender systems are generally trained and evaluated on samples of larger datasets. Samples are often taken in a naive or ad-hoc fashion: e.g. by sampling a dataset randomly or by selecting users or items with many interactions. As we demonstrate, commonly-used data sampling schemes can have significant consequences on algorithm performance. Following this observation, this paper makes three main contributions: (1) characterizing the effect of sampling on algorithm performance, in terms of algorithm and dataset characteristics (e.g. sparsity characteristics, sequential dynamics, etc.); (2) designing SVP-CF, which is a data-specific sampling strategy, that aims to preserve the relative performance of models after sampling, and is especially suited to long-tailed interaction data; and (3) developing an oracle, DATA-GENIE, which can suggest the sampling scheme that is most likely to preserve model performance for a given dataset. The main benefit of DATA-GENIE is that it will allow recommender system practitioners to quickly prototype and compare various approaches, while remaining confident that algorithm performance will be preserved, once the algorithm is retrained and deployed on the complete data. Detailed experiments show that using DATA-GENIE, we can discard upto 5x more data than any sampling strategy with the same level of performance.