OpenRaiser/Intern-Atlas huggingface dataset:
Improving Chinese Character Representation with Formation Graph Attention Network
Authors: Xiaosu Wang, Yun Xiong, Hao Niu, Jingwen Yue, Yangyong Zhu, Philip S. Yu
Venue: CIKM
Year: 2021

Chinese characters are often composed of subcharacter components which are also semantically informative, and the component-level internal semantic features of a Chinese character inherently bring with additional information that benefits the semantic representation of the character. Therefore, there have been several studies that utilized subcharacter component information (e.g. radical, fine-grained components and stroke n-grams) to improve Chinese character representation. However we argue that it has not been fully explored what would be the best way of modeling and encoding a Chinese character. For improving the representation of a Chinese character, existing methods introduce more component-level internal semantic features as well as more semantic irrelevant subcharacter component information, and these semantic irrelevant subcharacter component will be noisy for representing a Chinese character. Moreover, existing methods suffer from the inability of discriminating the importance of the introduced subcharacter components, accordingly they can not filter out introduced noisy subcharacter component information. In this paper, we first decompose Chinese characters into components according to their formations, then model a Chinese character and its decomposed components as a graph structure named Chinese character formation graph; Chinese character formation graph can reserve the azimuth relationship among subcharacter components, and be advantageous to explicitly model the component-level internal semantic features of a Chinese character. Furtherly, we propose a novel model Chinese Character Formation Graph Attention Network (FGAT) which is able to discriminate the importance of the introduced subcharacter components and extract component-level internal semantic features of a Chinese character efficiently. To demonstrate the effectiveness of our research, we have conducted extensive experiments. The experimental results show that our model achieves better results than state-of-the-art (SOTA) approaches.
Efficient Learning to Learn a Robust CTR Model for Web-scale Online Sponsored Search Advertising
Authors: Xin Wang, Peng Yang, Shaopeng Chen, Lin Liu, Lian Zhao, Jiacheng Guo, Mingming Sun, Ping Li
Venue: CIKM
Year: 2021

Click-through rate (CTR) prediction is crucial for online sponsored search advertising. Several successful CTR models have been adopted in the industry, including the regularized logistic regression (LR). Nonetheless, the learning process suffers from two limitations: 1) Feature crosses for high-order information may generate trillions of features, which are sparse for online learning examples; 2) Rapid changing of data distribution brings challenges to the accurate learning since the model has to perform a fast adaptation on the new data. Moreover, existing adaptive optimizers are ineffective in handling the sparsity issue for high-dimensional features. In this paper, we propose to learn an optimizer in a meta-learning scenario, where the optimizer is learned on prior data and can be easily adapted to the new data. We firstly build a low-dimensional feature embedding on prior data to encode the association among features. Then, the gradients on new data can be decomposed into the low-dimensional space, enabling the parameter update smoothed and relieving the sparsity. Note that this technology could be deployed into a distributed system to ensure efficient online learning on the trillions-level parameters. We conduct extensive experiments to evaluate the algorithm in terms of prediction accuracy and actual revenue. Experimental results demonstrate that the proposed framework achieves a promising prediction on the new data. The final online revenue is noticeably improved compared to the baseline. This framework was initially deployed in Baidu Search Ads (a.k.a. Phoenix Nest) in 2014 and is currently still being used in certain modules of Baidu's ads systems.
Influence Maximization in Multi-Relational Social Networks
Authors: Wei Wang, Haili Yang, Yuanfu Lu, Yuanhang Zou, Xu Zhang, Shuting Guo, Leyu Lin
Venue: CIKM
Year: 2021

Influence maximization (IM) is a classic problem, which aims to find a set of k users (called seed set) in a social network such that the expected number of users influenced by the seed users is maximized. Existing IM algorithms mainly focus on one-by-one influence diffusion among users with friendships. However, in addition to 1-to-1 friendships, 1-to-N group relations usually exist in real social platforms, which are seldom fully exploited by conventional methods. In this paper, with the real-world datasets in WeChat, the largest online social platform in China, we first study the IM problem in multi-relational social networks consisting of friendships and group relations, and propose a novel Generate&Extend framework to find influential seed users for product promotion. Specifically, to achieve a trade-off between effectiveness and efficiency, we present a truncated meta-seed generator to select a small number of users, which are influential with consideration of both friendships and group relations. More importantly, a structural seed extender is put forward to extend the meta-seed set, so as to encode the differentiated propagation structures between friendships and group relations. Extensive online/offline experiments on three real-world datasets demonstrate that Generate&Extend significantly outperforms the state of the arts. Our Generate&Extend has been deployed at WeChat for mini-program promoting, and severing more than 200 million users.
Neural Information Diffusion Prediction with Topic-Aware Attention Network
Authors: Hao Wang, Cheng Yang, Chuan Shi
Venue: CIKM
Year: 2021

Information diffusion prediction targets on forecasting how information items spread among a set of users. Recently, neural networks have been widely used in modeling information diffusion, owing to the great successes of deep learning. However, in real-world information diffusion scenarios, users are likely to have different behaviors to information items from different topics. Existing neural-based methods failed to model the topic-specific diffusion patterns and dependencies, which have been shown to be useful in conventional non-neural methods. In this paper, we propose Topic-aware Attention Network (TAN) to take advantage of both topic-specific diffusion modeling and deep learning techniques. We jointly model the text content of information items and cascade sequences by incorporating topical context and user/position dependencies into user representations via attention mechanisms. A time-decayed aggregation module is further employed to integrate user representations for cascade representations, which can encode the topic-specific diffusion dependencies independently. Experimental results on diffusion prediction tasks over three realistic cascade datasets show that our model can achieve a relative improvement up to 9% against the best performing baseline in terms of Hits@10.
Top-k Tree Similarity Join
Authors: Jianhua Wang, Jianye Yang, Wenjie Zhang
Venue: CIKM
Year: 2021

Tree similarity join is useful for analyzing tree structured data. The traditional threshold-based tree similarity join requires a similarity threshold, which is usually a difficult task for users. To remedy this issue, we advocate the problem of top-k tree similarity join. Given a collection of trees and a parameter k, the top-k tree similarity join aims to find k tree pairs with minimum tree edit distance (TED). Although we show that this problem can be resolved by utilizing the threshold-based join, the efficiency is unsatisfactory. In this paper, we propose an efficient algorithm, namely TopKTJoin, which generates the candidate tree pairs incrementally using an inverted index. We also derive TED lower bound for the unseen tree pairs. Together with TED value of the k-th best join result seen so far, we have a chance to terminate the algorithm early without missing any correct results. To further improve the efficiency, we propose two optimization techniques in terms of index structure and verification mechanism. We conduct comprehensive performance studies on real and synthetic datasets. The experimental results demonstrate that TopKTJoin significantly outperforms the baseline method.
Graph Structure Aware Contrastive Knowledge Distillation for Incremental Learning in Recommender Systems
Authors: Yuening Wang, Yingxue Zhang, Mark Coates
Venue: CIKM
Year: 2021

Personalized recommender systems are playing an increasingly important role for online services. Graph Neural Network (GNN) based recommender models have demonstrated a superior capability to model users' interests thanks to rich relational information encoded in graphs. However, with the ever-growing volume of online information and the high computational complexity of training GNNs, it is difficult to perform frequent updates to provide the most up-to-date recommendations. There have been several attempts towards training GNN models in an incremental fashion to enable faster training times and permit more frequent model updates using the latest training data. The main technique is knowledge distillation, which aims to allow model updates while preserving key aspects of the model that were learned from the historical data. In this work, we develop a novel Graph Structure Aware Contrastive Knowledge Distillation for Incremental Learning in recommender systems, which is tailored to focus on the rich relational information in the recommendation context. We combine the contrastive distillation formulation with intermediate layer distillation to inject layer-level supervision. We demonstrate the effectiveness of our proposed distillation framework for GNN based recommendation systems on four commonly used datasets, showing consistent improvement over state-of-the-art alternatives.
Task Assignment with Worker Churn Prediction in Spatial Crowdsourcing
Authors: Ziwei Wang, Yan Zhao, Xuanhao Chen, Kai Zheng
Venue: CIKM
Year: 2021

The pervasiveness of GPS-enabled devices and wireless communication technologies flourish the market of Spatial Crowdsourcing (SC), which consists of location-based tasks and requires workers to physically be at specific locations to complete them. In this work, we study the problem of Worker Churn based Task Assignment in SC, where tasks are to be assigned by considering workers' churn. In particular, we aim to achieve the highest total rewards of task assignments based on the worker churn prediction. To solve the problem, we propose a two-phase framework, which consists of a worker churn prediction phase and a task assignment phase. In the first phase, we use an LSTM-based model to extract the latent feelings of workers based on the historical data and then estimate the idle time intervals of workers. In the assignment phase, we design an efficient greedy algorithm and a Kuhn-Munkras (KM)-based algorithm that can achieve the optimal task assignment. Extensive experiments offer insight into the effectiveness and efficiency of the proposed solutions.
Improving Irregularly Sampled Time Series Learning with Time-Aware Dual-Attention Memory-Augmented Networks
Authors: Zhen Wang, Yang Zhang, Ai Jiang, Ji Zhang, Zhao Li, Jun Gao, Ke Li, Chenhao Lu, Zujie Ren
Venue: CIKM
Year: 2021

Irregularly, asynchronously and sparsely sampled multivariate time series (IASS-MTS) are characterized by sparse non-uniform time intervals between successive observations and different sampling rates amongst series. Those properties pose substantial challenges to mainstream machine learning models for learning complicated relations within and across IASS-MTS. This is because that most of the models assume that the time series in question are even, complete (fixed-dimensional features) and synchronous. To address these challenges, we present a novel time-aware Dual-Attention and Memory-Augmented Network (DAMA-Net). The proposed model can leverage both time irregularity, multi-sampling rates and global temporal patterns information inherent in IASS-MTS so as to learn more effective representations for improving prediction performance. Comprehensive experiments on real datasets show that the DAMA-Net outperforms the state-of-the-art methods in multivariate time series classification task.
Enhancing User Interest Modeling with Knowledge-Enriched Itemsets for Sequential Recommendation
Authors: Chunyang Wang, Yanmin Zhu, Haobing Liu, Wenze Ma, Tianzi Zang, Jiadi Yu
Venue: CIKM
Year: 2021

Sequential recommendation which aims to predict a user's next interaction based on his/her previous behaviors, has attracted great attention. Recent studies mainly employ deep recurrent neural networks or self-attention networks to capture dynamic user preferences. However, existing methods merely focus on modeling users' clear interests in interacted items. We argue that for an interaction, the user may also have ambiguous interests in items that are semantically related to the interacted one. For comprehensively capturing user preferences, it is beneficial to discover potential interests from historical interactions at a broader itemset level. Therefore, in this paper, we propose a knowledge graph enhanced sequential recommendation model namely KGIE, which focuses on enhancing user interest modeling with knowledge-enriched itemsets by incorporating the knowledge graph. Specifically, in addition to item-level interest modeling with interacted items, we further construct knowledge-enriched itemsets that are extracted via high-order knowledge associations with the interacted items. For capturing personalized itemset-level interests, we design an attentive aggregation unit to combine item embeddings considering both inherent and contextual personalization signals. Furthermore, to balance the contributions of both two levels of interest modeling, we adaptively learn high-level preference representations with a gating fusion unit. Extensive experiments on three real-world datasets demonstrate the superior performance beyond state-of-the-art methods and recommendation interpretability of our model.
Vector-Quantized Autoencoder With Copula for Collaborative Filtering
Authors: Guanyu Wang, Ting Zhong, Xovee Xu, Kunpeng Zhang, Fan Zhou, Yong Wang
Venue: CIKM
Year: 2021

In theory, the variational auto-encoder (VAE) is not suitable for recommendation tasks, although it has been successfully utilized for collaborative filtering (CF) models. In this paper, we propose a Gaussian Copula-Vector Quantized Autoencoder (GC-VQAE) model that differs prior arts in two key ways: (1) Gaussian Copula helps to model the dependencies among latent variables which are used to construct a more complex distribution compared with the mean-field theory; and (2) by incorporating a vector quantisation method into encoders our model can learn discrete representations which are consistent with the observed data rather than directly sampling from the simple Gaussian distributions. Our approach is able to circumvent the "posterior collapse'' issue and break the prior constraint to improve the flexibility of latent vector encoding and learning ability. Empirically, GC-VQAE can significantly improve the recommendation performance compared to existing state-of-the-art methods.
Fast k-NN Graph Construction by GPU based NN-Descent
Authors: Hui Wang, Wan-Lei Zhao, Xiangxiang Zeng, Jianye Yang
Venue: CIKM
Year: 2021

NN-Descent is a classic k-NN graph construction approach. It is still widely employed in machine learning, computer vision, and information retrieval tasks due to its efficiency and genericness. However, the current design only works well on CPU. In this paper, NN-Descent has been redesigned to adapt to the GPU architecture. A new graph update strategy called selective update is proposed. It reduces the data exchange between GPU cores and GPU global memory significantly, which is the processing bottleneck under GPU computation architecture. This redesign leads to full exploitation of the parallelism of the GPU hardware. In the meantime, the genericness, as well as the simplicity of NN-Descent, are well-preserved. Moreover, a procedure that allows to k-NN graph to be merged efficiently on GPU is proposed. It makes the construction of high-quality k-NN graphs for out-of-GPU-memory datasets tractable. Our approach is 100-250× faster than the single-thread NN-Descent and is 2.5-5× faster than the existing GPU-based approaches as we tested on million as well as billion scale datasets.
Jasmine: Exploring the Dependency-Aware Execution on Distributed Shared Memory
Authors: Xing Wei, Huiqi Hu, Xuan Zhou, Xuecheng Qi, Weining Qian, Jiang Wang, Aoying Zhou
Venue: CIKM
Year: 2021

Distributed shared memory abstraction can coordinate a cluster of machine nodes to empower performance-critical queries with the scalable memory space and abundant parallelism. But to deploy the query under such an abstraction, the general execution model just makes operators expressed as multiple subtasks and sequentially schedule them in parallel, while neglecting those vital dependencies between subtasks and data. In this paper, we conduct the in-depth researches about the issues (i.e., low CPU Utilization and poor data locality) raised by the ignorance of dependencies, and then propose a dependency-aware query execution model called Jasmine, which can (i) help users explicitly declare the dependencies and (ii) take these declared dependencies into the consideration of execution to address the issues. We invite our audience to use the rich graphical interfaces to interact with Jasmine to explore the dependency-aware query execution on distributed shared memory.
Predicting Instance Type Assertions in Knowledge Graphs Using Stochastic Neural Networks
Authors: Tobias Weller, Maribel Acosta
Venue: CIKM
Year: 2021

Instance type information is particularly relevant to perform reasoning and obtain further information about entities in knowledge graphs (KGs). However, during automated or pay-as-you-go KG construction processes, instance types might be incomplete or missing in some entities. Previous work focused mostly on representing entities and relations as embeddings based on the statements in the KG. While the computed embeddings encode semantic descriptions and preserve the relationship between the entities, the focus of these methods is often not on predicting schema knowledge, but on predicting missing statements between instances for completing the KG. To fill this gap, we propose an approach that first learns a KG representation suitable for predicting instance type assertions. Then, our solution implements a neural network architecture to predict instance types based on the learned representation. Results show that our representations of entities are much more separable with respect to their associations with classes in the KG, compared to existing methods. For this reason, the performance of predicting instance types on a large number of KGs, in particular on cross-domain KGs with a high variety of classes, is significantly better in terms of F1-score than previous work.
Evidential Relational-Graph Convolutional Networks for Entity Classification in Knowledge Graphs
Authors: Tobias Weller, Heiko Paulheim
Venue: CIKM
Year: 2021

Despite the vast amount of information encoded in knowledge graphs, they often remain incomplete. Neural networks, in particular Graph Convolutional Neural Networks, have been shown to be effective predictors to complete information about the class affiliation of entities in knowledge graphs. However, these models remain ignorant to their predictions confidence due to their used point estimate of a softmax output. In this paper, we combine Graph Convolutional Neural Networks with recent developments in the field of Evidential Learning by placing a Dirichlet distribution on the class probabilities to overcome this problem. We use the continuous output of a Graph Convolutional Neural Network as parameters for a Dirichlet distribution. In this way, the predictions of the model are represented as a distribution over possible softmax outputs, rather than a point estimate of a softmax output. The experiments show that a better performance in predicting class affiliations can be achieved compared to recent models. In addition, the experiments show that this approach overcomes the well-known problem of overconfident prediction of deterministic neural networks.
SMAD: Scalable Multi-view Ad Retrieval System for E-Commerce Sponsored Search
Authors: Shiyang Wen, Yiran Chen, Zhi Yang, Yan Zhang, Di Zhang, Liang Wang, Bo Zheng
Venue: CIKM
Year: 2021

Ad retrieval in sponsored search aims to understand user search intentions (user queries) and retrieves a set of ads inferred as being relevant to the queries. Due to the huge amount of search traffic and multiple views of relevance (such as co-clicking, co-bidding or textual similar), it is highly desirable but remain challenging to achieve a large-scale, multi-view matching between queries and ads, particularly in industrial settings. In this paper, we propose a scalable multi-view ad retrieval engine SMAD that we developed and deployed at Taobao, the largest e-commerce platform in China. We construct a multi-relation query-item-ad graph capturing different views of query-ad relevance, which is of large scale and with complex structure. Since in e-commerce platform, the queries and products are organized into a category tree, to deal with the large scale of the graph, we propose a category constrained graph sampling and partition method to enable distributed parallel offline training. To tackle the complex multi-view structure, we propose a multi-view parallel deep neural network (DNN) model to combine the information from different views in a principled way. According to offline experiments and online A/B tests, our framework significantly outperforms baselines in terms of relevance, coverage, and revenue.
Flexible O&M for Telecom Networks at Huawei: A Language Model-based Approach
Authors: Shuang Wu, Zhen Qin, Zhen Wang, Siwei Rao, Qiang Ye, Xingyue Quan, Guangjian Tian
Venue: CIKM
Year: 2021

Flexible operation and maintenance (O&M) is critical for telecommunication (telecom) service providers due to the ever-growing communication networks. Currently, most O&M operations still rely on rule-based strategies, which only cover limited scenarios and are costly to extend for novel applications as expert knowledge is intensively involved. To build a more flexible O&M system, we propose a language model to extract useful representations out of massive network signaling messages and use the representations to perform downstream O&M tasks. Given that a vanilla language model is not directly applicable for the structured signaling messages, we develop an expert-knowledge-inspired statistical approach to preprocess the messages and a hierarchical network architecture to extract message relations among different levels. Moreover, network messages in the real world are often contaminated, which can mislead the language model to learn incorrect message patterns. To mitigate data contamination, we propose a reverse training method that prevents the language model from learning the contaminated data. We collected hundreds of thousands of signaling message flows to train the proposed signaling language model and applied the trained model to O&M tasks. Offline experiments show that our proposed language model captures various signaling protocols and the extracted representations enable us to achieve expert-level performance in network anomaly detection and service recognition. Our language model has been deployed online at Huawei and significantly improved O&M efficiency.