OpenRaiser/Intern-Atlas huggingface dataset:
DA-Net: Distributed Attention Network for Temporal Knowledge Graph Reasoning
Authors: Kangzheng Liu, Feng Zhao, Hongxu Chen, Yicong Li, Guandong Xu, Hai Jin
Venue: CIKM
Year: 2022

Predicting future events in dynamic knowledge graphs has attracted significant attention. Existing work models the historical information in a holistic way, which achieves satisfactory performance. However, in real-world scenarios, the influence of historical information on future events is changing over time. Therefore, it is difficult to distinguish the historical information of different roles by invariably embedding historical entities with simple vector stacking. Furthermore, it is laborious to explicitly learn a distributed representation of each historical repetitive fact at different timestamps. This poses a challenge to the widely adopted codec-based architectures. In this paper, we propose a novel model for predicting future events, namely Distributed Attention Network (DA-Net). Rather than obtaining the fixed representations of historical events, DA-Net attempts to learn the distributed attention of future events on repetitive facts at different historical timestamps inspired by human cognitive theory. In human cognitive theory, when humans make a decision, similar historical events are replayed during memory recall. Based on memory, the original intention is adjusted according to their recent knowledge developments, making the action more reasonable to the context. Experiments on four benchmark datasets demonstrate a substantial improvement of DA-Net on multiple evaluation metrics.
Knowledge Distillation via Hypersphere Features Distribution Transfer
Authors: Boheng Liu, Tianrui Zhang, Ligang Miao
Venue: CIKM
Year: 2022

Knowledge distillation (KD) is a widely applicable DNN (Deep Neural Network) compression technology, which aims to transfer knowledge from a pretrained teacher neural network to a target student neural network. In practice, an enormous teacher is extracted through the compression of a neural network to train a relatively compact student. In general, current KD approaches mostly minimize divergence between the intermediate layers or logits of the teacher network and student network. However, these methods ignore important features distribution in the teacher network space, which leads to the defect of current KD approaches in the fine-grained categorization task, e.g., metric learning. For this, we propose a novel approach that transfers features distribution in the hyperspherical space from the teacher network to the student network. Specifically, our approach facilitates the student to learn the distribution among samples in the teacher and reduces the intra-class variance. Extensive experimental evaluations on three well-known metric learning datasets show that our method can distill higher-level knowledge from the teacher network and achieve state-of-the-art performance.
MomNet: Gender Prediction using Mechanism of Working Memory
Authors: Sijie Long, Lin Li, Jingling Yuan, Jianquan Liu
Venue: CIKM
Year: 2022

In social media analysis, gender prediction is one of the most important tasks of user profiling. Web users often post messages in a timeline manner to record their living moments. These messages containing texts and images, constitute long multi-modal data that potentially represents the living style, preference, or opinion regarding users. Therefore, it is feasible to predict the gender of a user by utilizing such living moments. However, the rich modalities (time, length, text, and image) of living moments with difficult challenges have not been fully exploited by the research communities for practical applications. To this end, we propose a novel gender prediction framework based on user-posted living Moments MomNet). The MomNet mainly consists of a moment memory module and a central executive module inspired by the two characteristics of working memory theory. One is that humans can associate related information to facilitate memory. Our moment memory module aggregates similar uni-modal moments of a user to form different chunks and encode the chunks into moment memory representations. The other is that humans coordinate information from different modalities to make judgments. Our central executive module is designed to coordinate comprehensive attentions of moment memory representations from texts, images, and their combinations. Finally, a softmax classifier is used to predict gender. Extensive experiments conducted on a real-world public dataset show that our framework achieves 86.63% accuracy and outperforms all state-of-the-art methods in terms of accuracy.
Meta-Reinforcement Learning for Multiple Traffic Signals Control
Authors: Yican Lou, Jia Wu, Yunchuan Ran
Venue: CIKM
Year: 2022

Despite the success of recent reinforcement learning (RL) in traffic signal control which has shown to outperform the conventional control methods, current RL-based methods require large amounts of samples to learn and lack the generalization ability to a new environment. In order to solve these problems, we propose a new context-based meta-RL model that disentangles task inference and control, which improves the meta-training efficiency and accelerates the learning process in a new environment. Moreover, the Graph Attention Network is employed to achieve effective cooperation between intersections. The experiments show that our method not only improves the traffic control efficiency but also converges faster and performs more stably, compared with traditional, RL-based, and meta-RL-based traffic control methods.
STARDOM: Semantic Aware Deep Hierarchical Forecasting Model for Search Traffic Prediction
Authors: Yucheng Lu, Qiang Ji, Liang Wang, Tianshu Wu, Hongbo Deng, Jian Xu, Bo Zheng
Venue: CIKM
Year: 2022

We study the search traffic forecasting problem for guaranteed search advertising (GSA) application in e-commerce platforms. The consumers express their purchase intents by posing queries to the e-commerce search engine. GSA is a type of guaranteed delivery (GD) advertising strategy, which forecasts the traffic of search queries, and charges the advertisers according to the predicted volumes of search queries the advertisers willing to buy. We employ the time series forecasting method to make the search traffic prediction. Different from existing time series prediction methods, search queries are semantically meaningful, with semantically similar queries possessing similar time series. And they can be grouped according to the brands or categories they belong to, exhibiting hierarchical structures. To fully take advantage of these characteristics, we design a SemanTic AwaRe Deep hierarchical fOrecasting Model (STARDOM for short) which explores the queries' semantic information and the hierarchical structures formed by the queries. Specifically, to exploit hierarchical structure, we propose a reconciliation learning module. It leverages deep learning model to learn the reconciliation relation between the hierarchical series in the latent space automatically, and forces the coherence constraints through a distill reconciliation loss. To exploit semantic information, we propose a semantic representation module and generate semantic aware series embeddings for queries. Extensive experiments are conducted to confirm the effectiveness of the proposed method.
PyKale: Knowledge-Aware Machine Learning from Multiple Sources in Python
Authors: Haiping Lu, Xianyuan Liu, Shuo Zhou, Robert Turner, Peizhen Bai, Raivo E. Koot, Mustafa Chasmai, Lawrence Schobs, Hao Xu
Venue: CIKM
Year: 2022

PyKale is a Python library for Knowledge-aware machine learning from multiple sources of data to enable/accelerate interdisciplinary research. It embodies green machine learning principles to reduce repetitions/redundancy, reuse existing resources, and recycle learning models across areas. We propose a pipeline-based application programming interface (API) so all machine learning workflows follow a standardized six-step pipeline. PyKale focuses on leveraging knowledge from multiple sources for accurate and interpretable prediction, particularly multimodal learning and transfer learning. To be more accessible, it separates code and configurations to enable non-programmers to configure systems without coding. PyKale is officially part of the PyTorch ecosystem and includes interdisciplinary examples in bioinformatics, knowledge graph, image/video recognition, and medical imaging: https://pykale.github.io/.
Scalable Multiple Kernel k-means Clustering
Authors: Yihang Lu, Haonan Xin, Rong Wang, Feiping Nie, Xuelong Li
Venue: CIKM
Year: 2022

With its simplicity and effectiveness, k-means is immensely popular, but it cannot perform well on complex nonlinear datasets. Multiple kernel k-means (MKKM) demonstrates the ability to describe highly complex nonlinear separable data structures. However, its speed requirement cannot scale as well as the data size grows beyond tens of thousands. Nowadays, digital data explosion mandates more scalable clustering methods to assist the machine learning tasks in easy-to-access form. To address the issue, we propose to employ the Nystrom scheme for MKKM clustering, termed scalable multiple kernel k-means clustering. It significantly reduces the computational complexity by replacing the original kernel matrix with a low-rank approximation. Analytically and empirically, we demonstrate that our method performs as well as existing state-of-the-art methods, but at a significantly lower compute cost, allowing us to scale the method more effectively for clustering tasks.
Self-Paced and Discrete Multiple Kernel k-Means
Authors: Yihang Lu, Xuan Zheng, Jitao Lu, Rong Wang, Feiping Nie, Xuelong Li
Venue: CIKM
Year: 2022

Multiple Kernel K-means (MKKM) uses various kernels from different sources to improve clustering performance. However, most of the existing models are non-convex, which is prone to be stuck into bad local optimum, especially with noise and outliers. To address the issue, we propose a novel Self-Paced and Discrete Multiple Kernel K-Means (SPD-MKKM). It learns the MKKM model in a meaningful order by progressing both samples and kernels from easy to complex, which is beneficial to avoid bad local optimum. In addition, whereas existing methods optimize in two stages: learning the relaxation matrix and then finding the discrete one by extra discretization, our work can directly gain the discrete cluster indicator matrix without extra process. What's more, a well-designed alternative optimization is employed to reduce the overall computational complexity via using the coordinate descent technique. Finally, thorough experiments performed on real-world datasets illustrated the excellence and efficacy of our method.
Learning Chinese Word Embeddings By Discovering Inherent Semantic Relevance in Sub-characters
Authors: Wei Lu, Zhaobo Zhang, Pingpeng Yuan, Hai Jin, Qiang-Sheng Hua
Venue: CIKM
Year: 2022

Learning Chinese word embeddings is important in many tasks of Chinese language information processing, such as entity linking, entity extraction, and knowledge graph. A Chinese word consists of Chinese characters, which can be decomposed into sub-characters (radical, component, stroke, etc). Similar to roots in English words, sub-characters also indicate the origins and basic semantics of Chinese characters. So, many researches follow the approaches designed for learning embeddings of English words to improve Chinese word embeddings. However, some Chinese characters sharing the same sub-characters have different meanings. Furthermore, with more cultural interaction and the popularization of the Internet and web, many neologisms, such as transliterated loanwords and network terms, are emerging, which are only close to the pronunciation of their characters, but far from their semantics. Here, a tripartite weighted graph is proposed to model the semantic relationship among words, characters, and sub-characters, in which the semantic relationship is evaluated according to the Chinese linguistic information. So, the semantic relevance hidden in lower components (sub-characters, characters) can be used to further distinguish the semantics of corresponding higher components (characters, words). Then, the tripartite weighted graph is fed into our Chinese word embedding modelinsideCC to reveal the semantic relationship among different language components, and learn the embeddings of words. Extensive experimental results on multiple corpora and datasets verify that our proposed methods outperform the state-of-the-art counterparts by a significant margin.
Dual-Task Learning for Multi-Behavior Sequential Recommendation
Authors: Jinwei Luo, Mingkai He, Xiaolin Lin, Weike Pan, Zhong Ming
Venue: CIKM
Year: 2022

Recently, sequential recommendation has become a research hotspot while multi-behavior sequential recommendation (MBSR) that exploits users' heterogeneous interactions in sequences has received relatively little attention. Existing works often overlook the complementary effect of different perspectives when addressing the MBSR problem. In addition, there are two specific challenges remained to be addressed. One is the heterogeneity of a user's intention and the context information, the other one is the sparsity of the interactions of target behavior. To release the potential of multi-behavior interaction sequences, we propose a novel framework named NextIP that adopts a dual-task learning strategy to convert the problem to two specific tasks, i.e., next-item prediction and purchase prediction. For next-item prediction, we design a target-behavior aware context aggregator (TBCG), which utilizes the next behavior to guide all kinds of behavior-specific item sub-sequences to jointly predict the next item. For purchase prediction, we design a behavior-aware self-attention (BSA) mechanism to extract a user's behavior-specific interests and treat them as negative samples to learn the user's purchase preferences. Extensive experimental results on two public datasets show that our NextIP performs significantly better than the state-of-the-art methods.
HySAGE: A Hybrid Static and Adaptive Graph Embedding Network for Context-Drifting Recommendations
Authors: Sichun Luo, Xinyi Zhang, Yuanzhang Xiao, Linqi Song
Venue: CIKM
Year: 2022

The recent popularity of edge devices and Artificial Intelligent of Things (AIoT) has driven a new wave of contextual recommendations, such as location based Point of Interest (PoI) recommendations and computing resource-aware mobile app recommendations. In many such recommendation scenarios, contexts are drifting over time. For example, in a mobile game recommendation, contextual features like locations, battery, and storage levels of mobile devices are frequently drifting over time. However, most existing graph-based collaborative filtering methods are designed under the assumption of static features. Therefore, they would require frequent retraining and/or yield graphical models burgeoning in sizes, impeding their suitability for context-drifting recommendations. In this work, we propose a specifically tailor-made Hybrid Static and Adaptive Graph Embedding (HySAGE) network for context-drifting recommendations. Our key idea is to disentangle the relatively static user-item interaction and rapidly drifting contextual features. Specifically, our proposed HySAGE network learns a relatively static graph embedding from user-item interaction and an adaptive embedding from drifting contextual features. These embeddings are incorporated into an interest network to generate the user interest in some certain context. We adopt an interactive attention module to learn the interactions among static graph embeddings, adaptive contextual embeddings, and user interest, helping to achieve a better final representation. Extensive experiments on real-world datasets demonstrate that HySAGE significantly improves the performance of the existing state-of-the-art recommendation algorithms.
See Clicks Differently: Modeling User Clicking Alternatively with Multi Classifiers for CTR Prediction
Authors: Shiwei Lyu, Hongbo Cai, Chaohe Zhang, Shuai Ling, Yue Shen, Xiaodong Zeng, Jinjie Gu, Guannan Zhang, Haipeng Zhang
Venue: CIKM
Year: 2022

Many recommender systems optimize click through rates (CTRs) as one of their core goals, and it further breaks down to predicting each item's click probability for a user (user-item click probability) and recommending the top ones to this particular user. User-item click probability is then estimated as a single term, and the basic assumption is that the user has different preferences over items. This is presumably true, but from real-world data, we observe that some people are naturally more active in clicking on items while some are not. This intrinsic tendency contributes to their user-item click probabilities. Besides this, when a user sees a particular item she likes, the click probability for this item increases due to this user-item preference. Therefore, instead of estimating the user-item click probability directly, we break it down into two finer attributes: user's intrinsic tendency of clicking and user-item preference. Inspired by studies that emphasize item features for overall enhancements and research progress in multi-task learning, we for the first time design a Multi Classifier Click Rate prediction model (MultiCR) to better exploit item-level information by building a separate classifier for each item. Furthermore, in addition to utilizing static user features, we learn implicit connections between user's item preferences and the often-overlooked indirect user behaviors (e.g., click histories from other services within the app). In a common new-campaign/new-service scenario, MultiCR outperforms various baselines in large-scale offline and online experiments and demonstrates good resilience when the amount of training data decreases.
Faithful Abstractive Summarization via Fact-aware Consistency-constrained Transformer
Authors: Yuanjie Lyu, Chen Zhu, Tong Xu, Zikai Yin, Enhong Chen
Venue: CIKM
Year: 2022

Abstractive summarization is a classic task in Natural Language Generation (NLG), which aims to produce a concise summary of the original document. Recently, great efforts have been made on sequence-to-sequence neural networks to generate abstractive sum- maries with a high level of fluency. However, prior arts mainly focus on the optimization of token-level likelihood, while the rich semantic information in documents has been largely ignored. In this way, the summarization results could be vulnerable to hallucinations, i.e., the semantic-level inconsistency between a summary and corresponding original document. To deal with this challenge, in this paper, we propose a novel fact-aware abstractive summarization model, named Entity-Relation Pointer Generator Network (ERPGN). Specially, we attempt to formalize the facts in original document as a factual knowledge graph, and then generate the high-quality summary via directly modeling consistency between summary and the factual knowledge graph. To that end, we first leverage two pointer net- work structures to capture the fact in original documents. Then, to enhance the traditional token-level likelihood loss, we design two extra semantic-level losses to measure the disagreement between a summary and facts from its original document. Extensive experi- ments on public datasets demonstrate that our ERPGN framework could outperform both classic abstractive summarization models and the state-of-the-art fact-aware baseline methods, with significant improvement in terms of faithfulness.
Towards Fair Workload Assessment via Homogeneous Order Grouping in Last-mile Delivery
Authors: Wenjun Lyu, Kexin Zhang, Baoshen Guo, Zhiqing Hong, Guang Yang, Guang Wang, Yu Yang, Yunhuai Liu, Desheng Zhang
Venue: CIKM
Year: 2022

The popularity of e-commerce has promoted the rapid development of the logistics industry in recent years. As an important step in logistics, last-mile delivery from delivery stations to customers' addresses is now mainly finished by couriers, which requires accurate workload assessment based on actual efforts. However, the state-of-the-practice assessment methods neglect a vital factor that orders with the same customer's address (i.e., Homogeneous orders) can be delivered in a group (i.e., in a single trip) or separately (i.e., in multiple trips). It would cause unfair assessment among couriers if following the same rule. Thus, grouping homogeneous order accurately in the workload assessment is significant for achieving fair courier's workload assessment. To this end, we design, implement, and deploy a nationwide homogeneous order grouping system called FHOG for improving the accuracy of homogeneous order grouping in last-mile delivery for fair courier's workload assessment. FHOG utilizes the courier's reporting behavior for order inspection, collection, and delivery to identify homogeneous orders in the delivery station simultaneously for homogeneous order grouping. Compared with the state-of-the-practice method, our evaluation shows FHOG can effectively reduce order amounts with the higher and lower assessed courier's workload. We further deploy FHOG online in 8336 delivery stations to provide homogeneous order grouping service for more than 120 thousand couriers and 12 million daily orders. The results of the two surveys show that the couriers' acceptance rate is improved by 67% with FHOG after the promotion.
Hierarchical Spatio-Temporal Graph Neural Networks for Pandemic Forecasting
Authors: Yihong Ma, Patrick Gérard, Yijun Tian, Zhichun Guo, Nitesh V. Chawla
Venue: CIKM
Year: 2022

The spread of COVID-19 throughout the world has led to cataclysmic consequences on the global community, which poses an urgent need to accurately understand and predict the trajectories of the pandemic. Existing research has relied on graph-structured human mobility data for the task of pandemic forecasting. To perform pandemic forecasting of COVID-19 in the United States, we curate Large-MG, a large-scale mobility dataset that contains 66 dynamic mobility graphs, with each graph having over 3k nodes and an average of 540k edges. One drawback with existing Graph Neural Networks (GNNs) for pandemic forecasting is that they generally perform information propagation in a flat way and thus ignore the inherent community structure in a mobility graph. To bridge this gap, we propose a Hierarchical Spatio-Temporal Graph Neural Network (HiSTGNN) to perform pandemic forecasting, which learns both spatial and temporal information from a sequence of dynamic mobility graphs. HiSTGNN consists of two network architectures. One is a hierarchical graph neural network (HiGNN) that constructs a two-level neural architecture: county-level and region-level, and performs information propagation in a hierarchical way. The other network architecture is a Transformer-based model that captures the temporal dynamics among the sequence of learned node representations from HiGNN. Additionally, we introduce a joint learning objective to further optimize HiSTGNN. Extensive experiments have demonstrated HiSTGNN's superior predictive power of COVID-19 new case/death counts compared with state-of-the-art baselines.