OpenRaiser/Intern-Atlas huggingface dataset:
Attention Calibration for Transformer-based Sequential Recommendation
Authors: Peilin Zhou, Qichen Ye, Yueqi Xie, Jingqi Gao, Shoujin Wang, Jae Boum Kim, Chenyu You, Sunghun Kim
Venue: CIKM
Year: 2023

Transformer-based sequential recommendation (SR) has been booming in recent years, with the self-attention mechanism as its key component. Self-attention has been widely believed to be able to effectively select those informative and relevant items from a sequence of interacted items for next-item prediction via learning larger attention weights for these items. However, this may not always be true in reality. Our empirical analysis of some representative Transformer-based SR models reveals that it is not uncommon for large attention weights to be assigned to less relevant items, which can result in inaccurate recommendations. Through further in-depth analysis, we find two factors that may contribute to such inaccurate assignment of attention weights:sub-optimal position encoding andnoisy input. To this end, in this paper, we aim to address this significant yet challenging gap in existing works. To be specific, we propose a simple yet effective framework called Attention Calibration for Transformer-based Sequential Recommendation (AC-TSR). In AC-TSR, a novel spatial calibrator and adversarial calibrator are designed respectively to directly calibrates those incorrectly assigned attention weights. The former is devised to explicitly capture the spatial relationships (i.e., order and distance) among items for more precise calculation of attention weights. The latter aims to redistribute the attention weights based on each item's contribution to the next-item prediction. AC-TSR is readily adaptable and can be seamlessly integrated into various existing transformer-based SR models. Extensive experimental results on four benchmark real-world datasets demonstrate the superiority of our proposed AC-TSR via significant recommendation performance enhancements. The source code is available at https://github.com/AIM-SE/AC-TSR.
Detecting Social Bot on the Fly using Contrastive Learning
Authors: Ming Zhou, Dan Zhang, Yuandong Wang, Yangli-ao Geng, Jie Tang
Venue: CIKM
Year: 2023

Social bot detection is becoming a task of wide concern in social security. All along, the development of social bot detection technology is hindered by the lack of high-quality annotated data. Besides, the rapid development of AI Generated Content (AIGC) technology is dramatically improving the creative ability of social bots. For example, the recently released ChatGPT [2] can fool the state-of-the-art AI-text-detection method with a probability of 74%, bringing a large challenge to content-based bot detection methods. To address the above drawbacks, we propose a Contrastive Learning-driven Social Bot Detection framework (CBD). The core of CBD is characterized by a two-stage model learning strategy: a contrastive pre-training stage to mine generalization patterns from massive unlabeled social graphs, followed by a semi-supervised fine-tuning stage to model task-specific knowledge latent in social graphs with a few annotations. The above strategy endows our model with promising detection performance under an extreme scarcity of labeled data. In terms of system architecture, we propose a smart feedback mechanism to further improve detection performance. Comprehensive experiments on a real bot detection dataset show that CBD consistently outperforms 10 state-of-the-art baselines by a large margin for few-shot bot detection using very little (5-shot) labeled data. CBD has been deployed online.
FVW: Finding Valuable Weight on Deep Neural Network for Model Pruning
Authors: Zhiyu Zhu, Huaming Chen, Zhibo Jin, Xinyi Wang, Jiayu Zhang, Minhui Xue, Qinghua Lu, Jun Shen, Kim-Kwang Raymond Choo
Venue: CIKM
Year: 2023

The rapid development of deep learning has demonstrated its potential for deployment in many intelligent service systems. However, some issues such as optimisation (e.g., how to reduce the deployment resources costs and further improve the detection speed), especially in scenarios where limited resources are available, remain challenging to address. In this paper, we aim to delve into the principles of deep neural networks, focusing on the importance of network neurons. The goal is to identify the neurons that exert minimal impact on model performances, thereby aiding in the process of model pruning. In this work, we have thoroughly considered the deep learning model pruning process with and without fine-tuning step, ensuring the model performance consistency. To achieve our objectives, we propose a methodology that employs adversarial attack methods to explore deep neural network parameters. This approach is combined with an innovative attribution algorithm to analyse the level of network neurons involvement. In our experiments, our approach can effectively quantify the importance of network neuron. We extend the evaluation through comprehensive experiments conducted on a range of datasets, including CIFAR-10, CIFAR-100 and Caltech101. The results demonstrate that, our method have consistently achieved the state-of-the-art performance over many existing methods. We anticipate that this work will help to reduce the heavy training and inference cost of deep neural network models where a lightweight deep learning enhanced service and system is possible. The source code is open source at https://github.com/LMBTough/FVW.
Scalable Neural Contextual Bandit for Recommender Systems
Authors: Zheqing Zhu, Benjamin Van Roy
Venue: CIKM
Year: 2023

High-quality recommender systems ought to deliver both innovative and relevant content through effective and exploratory interactions with users. Yet, supervised learning-based neural networks, which form the backbone of many existing recommender systems, only leverage recognized user interests, falling short when it comes to efficiently uncovering unknown user preferences. While there has been some progress with neural contextual bandit algorithms towards enabling online exploration through neural networks, their onerous computational demands hinder widespread adoption in real-world recommender systems. In this work, we propose a scalable sample-efficient neural contextual bandit algorithm for recommender systems. To do this, we design an epistemic neural network architecture, Epistemic Neural Recommendation (ENR), that enables Thompson sampling at a large scale. In two distinct large-scale experiments with real-world tasks, ENR significantly boosts click-through rates and user ratings by at least 9% and 6% respectively compared to state-of-the-art neural contextual bandit algorithms. Furthermore, it achieves equivalent performance with at least 29% fewer user interactions compared to the best-performing baseline algorithm. Remarkably, while accomplishing these improvements, ENR demands orders of magnitude fewer computational resources than neural contextual bandit baseline algorithms.
PCENet: Psychological Clues Exploration Network for Multimodal Personality Assessment
Authors: Yangfu Zhu, Yuting Wei, Meiling Li, Tingting Zhang, Siqi Wei, Bin Wu
Venue: CIKM
Year: 2023

Multimodal personality assessment aims to identify and express human personality traits in videos. Existing methods primarily focus on multimodal fusion while ignoring the inherent psychological clues essential for this interdisciplinary task. Modality clues: personality traits are stable over time due to their genetic and environmental origins, resulting in stable personality traits in the multimodal data. Trait clues: multiple traits often co-occur with non-negligible correlations, which can collectively aid trait identification. To simultaneously capture the above psychological clues, we propose a novel Psychological Clues Exploration Network (PCENet) for multimodal personality assessment, which is a human-like judgment paradigm with more generalization capability. Specifically, we first devise a multimodal hierarchical disentanglement, which clearly aligns stable representations among different modalities and separates the mutability of each modality. Subsequently, a Transformer-backbone decoder equipped with modality-to-trait attention is exploited to adaptively generate a tailored representation for each trait with the guidance of trait semantics. The trait semantics are obtained by exploiting trait correlations through self-attention. Extensive experiments on the First Impression V2 dataset demonstrate that our PCENet outperforms the state-of-the-art methods for multimodal personality assessment.
CANDY: A Causality-Driven Model for Hotel Dynamic Pricing
Authors: Ruitao Zhu, Wendong Xiao, Yao Yu, Yizhi Yu, Zhenzhe Zheng, Ke Bu, Dong Li, Fan Wu
Venue: CIKM
Year: 2023

Broad adoption of online travel platforms (OTPs) has led to increasing focus on hotel dynamic pricing algorithms, which directly affect the revenue of platform and hotels. Existing approaches, which directly model the correlation between price and occupancy, have limitations in improving occupancy prediction accuracy while ensuring interpretability for dynamic pricing. Moreover, these methods struggle to address the significant data sparsity issue in hotel pricing scenarios. To overcome these limitations, we propose a novel Causality-driven Hotel Dynamic Pricing Model (CANDY) that captures the essential causal relationship between price and occupancy, enhancing occupancy prediction accuracy and interpretability for dynamic pricing. Specifically, we decompose confounders into three orthogonal groups of factors: characteristic factors, competitive factors, and temporal factors, and design submodules to capture the features of each dimension. To address the treatment bias and sample imbalance issues faced by existing causal inference methods in hotel pricing scenarios, we propose a novel data augmentation method based on the monotonic relationship between price and occupancy, and further design a multi-task learning framework tailored to multi-valued treatment scenarios, simultaneously alleviating the data sparsity issue. Both offline and online experiments demonstrate the effectiveness of CANDY in occupancy prediction and dynamic pricing. CANDY has been successfully deployed to provide price suggestion service at Fliggy, a leading OTP in China, serving thousands of hotel operators.
HCL4QC: Incorporating Hierarchical Category Structures Into Contrastive Learning for E-commerce Query Classification
Authors: Lvxing Zhu, Kexin Zhang, Hao Chen, Chao Wei, Weiru Zhang, Haihong Tang, Xiu Li
Venue: CIKM
Year: 2023

Query classification plays a crucial role in e-commerce, where the goal is to assign user queries to appropriate categories within a hierarchical product category taxonomy. However, existing methods rely on a limited number of words from the category description and often neglect the hierarchical structure of the category tree, resulting in suboptimal category representations. To overcome these limitations, we propose a novel approach named hierarchical contrastive learning framework for query classification (HCL4QC), which leverages the hierarchical category tree structure to improve the performance of query classification. Specifically, HCL4QC is designed as a plugin module that consists of two innovative losses, namely local hierarchical contrastive loss (LHCL) and global hierarchical contrastive loss (GHCL). LHCL adjusts representations of categories according to their positional relationship in the hierarchical tree, while GHCL ensures the semantic consistency between the parent category and its child categories. Our proposed method can be adapted to any query classification tasks that involve a hierarchical category structure. We conduct experiments on two real-world datasets to demonstrate the superiority of our hierarchical contrastive learning. The results demonstrate significant improvements in the query classification task, particularly for long-tail categories with sparse supervised information.
G-STO: Sequential Main Shopping Intention Detection via Graph-Regularized Stochastic Transformer
Authors: Yuchen Zhuang, Xin Shen, Yan Zhao, Chaosheng Dong, Ming Wang, Jin Li, Chao Zhang
Venue: CIKM
Year: 2023

Sequential recommendation requires understanding the dynamic patterns of users' behaviors, contexts, and preferences from their historical interactions. While most research emphasizes item-level user-item interactions, they often overlook underlying shopping intentions, such as preferences for ballpoint pens or miniatures. Identifying these latent intentions is vital for enhancing shopping experiences on platforms like Amazon. Despite its significance, the area of main shopping intention detection remains under-investigated in the academic literature. To fill this gap, we introduce a graph-regularized stochastic Transformer approach, G-STO. It considers intentions as product sets and user preferences as intention composites, both modeled as stochastic Gaussian embeddings in latent space. We also employ a global intention relational graph as prior knowledge for regularization, ensuring related intentions are distributionally close. These regularized embeddings are then input into Transformer-based models to capture sequential intention transitions. On testing our model with three real-world datasets, it outperformed the baselines by 18.08% in Hit@1, 7.01% in Hit@10, and 6.11% in NDCG@10.
An AI-based Simulation and Optimization Framework for Logistic Systems
Authors: Zefang Zong, Huan Yan, Hongjie Sui, Haoxiang Li, Peiqi Jiang, Yong Li
Venue: CIKM
Year: 2023

Improving logistics efficiency is a challenging task in logistic systems, since planning the vehicle routes highly relies on the changing traffic conditions and diverse demand scenarios. However, most existing approaches either neglect the dynamic traffic environment or adopt manually designed rules, which fails to efficiently find a high-quality routing strategy. In this paper, we present a novel artificial intelligence (AI) based framework for logistic systems. This framework can simulate the spatio-temporal traffic conditions to form a dynamic environment in a data-driven manner. Under such a simulated environment, it adopts deep reinforcement learning techniques to intelligently generate the optimized routing strategy. Meanwhile, we also design an interactive frontend to visualize the simulated environment and routing strategies, which help operators evaluate the task performance. We will showcase the results of AI-based simulation and optimization in our demonstration.
Product Entity Matching via Tabular Data
Authors: Ali Naeim abadi, Mir Tafseer Nayeem, Davood Rafiei
Venue: CIKM
Year: 2023

Product Entity Matching (PEM)--a subfield of record linkage that focuses on linking records that refer to the same product--is a challenging task for many entity matching models. For example, recent transformer models report a near-perfect performance score on many datasets while their performance is the lowest on PEM datasets. In this paper, we study PEM under the common setting where the information is spread over text and tables. We show that adding tables can enrich the existing PEM datasets and those tables can act as a bridge between the entities being matched. We also propose TATEM, an effective solution that leverages Pre-trained Language Models (PLMs) with a novel serialization technique to encode tabular product data and an attribute ranking module to make our model more data-efficient. Our experiments on both current benchmark datasets and our proposed datasets show significant improvements compared to state-of-the-art methods, including Large Language Models (LLMs) in zero-shot and few-shot settings.
Seeing the Forest for the Trees: Road-Level Insights Assisted Lane-Level Traffic Prediction
Authors: Shuhao Li, Yue Cui, Jingyi Xu, Jing Zhao, Fan Zhang, Weidong Yang, Xiaofang Zhou
Venue: CIKM
Year: 2024

Lane-level traffic prediction is crucial for refined smart city applications, yet the scarcity and quality issues of datasets hinder its development. To overcome these challenges, this study introduces a novel M ulti- c hannel g raph-structured V ariational A uto E ncoder model, McgVAE. This model integrates road-level information to provide a global perspective for lane prediction and performs integrated tasks through three interconnected channels: the road-level channel ensures accurate prediction of road traffic states and communicates closely with the data quality channel to share historical and predicted road information; the data quality channel leverages road-level information to identify and correct missing and noisy data; and finally, the lane channel uses the aforementioned information for lane-level traffic prediction. After extensive experimental comparisons with multiple baseline models across three datasets, the McgVAE model demonstrated outstanding predictive performance and the ability to handle data missingness and noise
Inferring Visualization Intent from Conversation
Authors: Haotian Li, Nithin Chalapathi, Huamin Qu, Alvin Cheung, Aditya G. Parameswaran
Venue: CIKM
Year: 2024

During visual data analysis, users often explore visualizations one at a time, with each visualization leading to new directions of exploration. We consider a conversational approach to visualization, where users specify their needs at each step in natural language, with a visualization being returned in turn. Prior work has shown that visualization generation can be boiled down to the identification of visualization intent and visual encodings. Recognizing that the latter is a well-studied problem with standard solutions, we focus on the former, i.e., identifying visualization intent during conversation. We develop Luna, a framework that comprises a novel combination of language models adapted from BERT and rule-based inference, that together predict various aspects of visualization intent. We compare Luna with other conversational NL-to-visualization and NL-to-SQL approaches (adapted to visualization intent), including GPT-3.5 and GPT-4, and demonstrate that Luna has 14.3% higher accuracy than the state-of-the-art. We also apply Luna to a usage scenario on a dataset of police misconduct, showcasing its benefits relative to other approaches.
Information Retrieval Optimization for Non-Exemplar Class Incremental Learning
Authors: Shuai Guo, Yang Gu, Yuan Ma, Yingwei Zhang, Weining Weng, Jun Liu, Weiwei Dai, Yiqiang Chen
Venue: CIKM
Year: 2024

Existing non-example class-incremental learning (NECIL) methods usually utilize a combination strategy of replay mechanism and knowledge distillation. However, this combination strategy only focuses on the preservation of old information quantitatively, ignoring the preservation quality. When the old knowledge has wrong redundant information, catastrophic forgetting is more likely to occur. Therefore, obtaining adequate information without impurities as much as possible and removing invalid or even harmful information has become an effective solution to improve the performance of NECIL. This process is consistent with the information bottleneck (IB) theory. Thus, we propose a new NECIL method based on the IB framework. By using the different information obtained from the new and old class samples and the implicit knowledge in the teacher model training process, the error of harmful redundant information learned is eliminated. Specifically, we propose two optimization strategies that align with the two optimization processes of the information bottleneck. Firstly, we employ a pseudo-prototype selection mechanism that selectively incorporates pseudo-samples into the learning process of new and old categories, thus enhancing the distinction between new and old categories and diminishing the mutual information between the input and intermediate features. Secondly, we introduce an attention-based feature distillation method that regulates the distillation strength between feature pairs based on their similarity, thereby augmenting the mutual information between intermediate features and output prediction. Extensive experiments on three benchmarks demonstrate that the proposed method exhibits significant incremental performance improvements over existing methods.
Mining Path Association Rules in Large Property Graphs
Authors: Yuya Sasaki, Panagiotis Karras
Venue: CIKM
Year: 2024

How can we mine frequent path regularities from a graph with edge labels and vertex attributes? The task of association rule mining successfully discovers regular patterns in item sets and substructures. Still, to our best knowledge, this concept has not yet been extended to path patterns in large property graphs. In this paper, we introduce the problem of path association rule mining (PARM). Applied to any reachability path between two vertices within a large graph, PARM discovers regular ways in which path patterns, identified by vertex attributes and edge labels, co-occur with each other. We develop an efficient and scalable algorithm PIONEER that exploits an anti-monotonicity property to effectively prune the search space. Further, we devise approximation techniques and employ parallelization to achieve scalable path association rule mining. Our experimental study using real-world graph data verifies the significance of path association rules and the efficiency of our solutions.
Adaptive Cascading Network for Continual Test-Time Adaptation
Authors: Kien X. Nguyen, Fengchun Qiao, Xi Peng
Venue: CIKM
Year: 2024

We study the problem of continual test-time adaption where the goal is to adapt a source pre-trained model to a sequence of unlabelled target domains at test time. Existing methods on test-time training suffer from several limitations: (1) Mismatch between the feature extractor and classifier; (2) Interference between the main and self-supervised tasks; (3) Lack of the ability to quickly adapt to the current distribution. In light of these challenges, we propose a cascading paradigm that simultaneously updates the feature extractor and classifier at test time, mitigating the mismatch between them and enabling long-term model adaptation. The pre-training of our model is structured within a meta-learning framework, thereby minimizing the interference between the main and self-supervised tasks and encouraging fast adaptation in the presence of limited unlabelled data. Additionally, we introduce innovative evaluation metrics, average accuracy and forward transfer, to effectively measure the model's adaptation capabilities in dynamic, real-world scenarios. Extensive experiments and ablation studies demonstrate the superiority of our approach in a range of tasks including image classification, text classification, and speech recognition. Our code is publicly available at https://github.com/Nyquixt/CascadeTTA.
Revisiting Optimal Window Aggregation in Data Streams: The Prefix-Sum Approach
Authors: José Martinez, Guillaume Raschia
Venue: CIKM
Year: 2024

The article presents a simple yet optimal approach to compute aggregates of window queries over data streams. The proposal is built as a fully-fledged pipeline of operators that are literal transcripts of mathematical definitions. Main features are an application of the prefix sums and a well-founded un-/slicing technique. The overall process is linear in the number of events and windows, and it takes quasi-constant space for a mix of periodic windows, still applicable to multiple deterministic window queries. The limitations are twofold: the events come in-order and the aggregation function is a left-cancellative monoid.