OpenRaiser/Intern-Atlas huggingface dataset:
Fast and Accurate PARAFAC2 Decomposition for Time Range Queries on Irregular Tensors
Authors: Jun-Gi Jang, Yong-chan Park, U Kang
Venue: CIKM
Year: 2024

How can we efficiently analyze a specific time range on an irregular tensor? PARAFAC2 decomposition is widely used when analyzing an irregular tensor which consists of several matrices with different row sizes. A crucial task related to PARAFAC2 decomposition is to analyze sub-tensors corresponding to various time ranges of a given tensor, instead of analyzing the entire tensor. Although many recent works have developed efficient PARAFAC2 decomposition methods, existing PARAFAC2 decomposition methods are inappropriate for addressing various time range queries, as they need to decompose sub-tensors from scratch. In this paper, we propose Repeat, a fast and accurate PARAFAC2 decomposition method for handling arbitrary time range queries on irregular tensors. To avoid decomposing sub-tensors of queries from scratch, Repeat obtains preprocessed results that support efficient query answering before time ranges are given. For time range queries, Repeat efficiently computes the PARAFAC2 decomposition for the sub-tensors corresponding to the queries by using preprocessed results rather than the original irregular tensor. We experimentally demonstrate that Repeat outperforms existing PARAFAC2 methods, providing up to 12x faster speed while having comparable errors. We also present a case study for the use of Repeat in detecting locally appearing patterns through a variety of time range queries.
Evaluating Social Media Reach via Mainstream Media Discourse
Authors: Himarsha R. Jayanetti
Venue: CIKM
Year: 2024

This study examines the intersection between social media and mainstream television (TV) news with an aim to understand how social media content amplifies its impact through TV broadcasts. While many studies emphasize social media as a primary platform for information dissemination, they often underestimate its total influence by focusing solely on interactions within the platform. This research examines instances where social media posts gain prominence on TV broadcasts, reaching new audiences and prompting public discourse. By using TV news closed captions, on-screen text recognition, and social media logo detection, we analyze how social media is referenced in TV news. Our methodology aims to analyze this data to develop metrics that quantify the extent of this amplification and understand the contexts in which social media is integrated into broadcast content.
Calibration-Disentangled Learning and Relevance-Prioritized Reranking for Calibrated Sequential Recommendation
Authors: Hyunsik Jeon, Se-eun Yoon, Julian J. McAuley
Venue: CIKM
Year: 2024

Calibrated recommendation, which aims to maintain personalized proportions of categories within recommendations, is crucial in practical scenarios since it enhances user satisfaction by reflecting diverse interests. However, achieving calibration in a sequential setting (i.e., calibrated sequential recommendation) is challenging due to the need to adapt to users' evolving preferences. Previous methods typically leverage reranking algorithms to calibrate recommendations after training a model without considering the effect of calibration and do not effectively tackle the conflict between relevance and calibration during the reranking process. In this work, we propose LeapRec (Calibration-Disentangled Learning and Relevance-Prioritized Reranking), a novel approach for the calibrated sequential recommendation that addresses these challenges. LeapRec consists of two phases, model training phase and reranking phase. In the training phase, a backbone model is trained using our proposed calibration-disentangled learning-to-rank loss, which optimizes personalized rankings while integrating calibration considerations. In the reranking phase, relevant items are prioritized at the top of the list, with items needed for calibration following later to address potential conflicts between relevance and calibration. Through extensive experiments on four real-world datasets, we show that LeapRec consistently outperforms previous methods in the calibrated sequential recommendation. Our code is available at https://github.com/jeon185/LeapRec.
HiLite: Hierarchical Level-implemented Architecture Attaining Part-Whole Interpretability
Authors: Yoo Hyun Jeong, Sunghyun Hwang, Dong-Kyu Chae
Venue: CIKM
Year: 2024

Beyond the traditional CNN structure, we have recently witnessed lots of breakthroughs in computer vision architectures such as Vision Transformer, MLP-Mixer, SNN-MLP, and so on. However, many efforts in developing novel architectures for vision tasks are heavily focused on achieving powerful performances, and how to attain interpretability in a trained neural network remains an open question. Inspired by the imaginary system GLOM, we present HiLite : Hierarchical Level-implemented Architecture attaining Part-Whole Interpretability, where islands of identical vectors can provide unprecedented interpretability. In our column-like structure, each level is a layer of a part-whole hierarchy composed of multiple neurons, and the function to define the neural field along an image input patch is initialized as the level vector inside the model. We propose two-column networks (Top-Down (TD) and Bottom-Up (BU)) that allow inter-level communication between adjacent levels on a specific patch and propose Gated Consensus Attention to perform intra-level communication on different patches within the level. At each time step, the level vector and outputs from different networks are combined into a weighted sum and passed to the next step, and outputs from the final time step are utilized as representation vectors. Here, supervised contrastive learning is used to find the relationship of meaningful patches in each class, where negative examples contribute to preventing representation collapse between neighboring patches. HiLite shows a possibility of performance through a quantitative evaluation on four image classification datasets as well as two metrics for assessing representation quality and showcases the intrinsic interpretability by simply generating a visual cue. We believe that our work is a solid step towards novel research on neural architectures attaining interpretability.
GraphVAE: Unveiling Dynamic Stock Relationships with Variational Autoencoder-based Factor Modeling
Authors: Yulong Jia, Guanxing Li, Ganlong Zhao, Xiangru Lin, Guanbin Li
Venue: CIKM
Year: 2024

Factor models, originating in finance for asset pricing, are fundamental tools in quantitative investment. Recently, there has been a trend towards adopting more flexible machine learning approaches instead of previous linear models. However, traditional factor models and recent deep learning approaches either overlook the relationships among stocks or rely on static, predefined ones, which hampers their representational power and hinders their ability to dynamically adapt to market changes. To overcome this limitation, we introduce a novel dynamic factor model named GraphVAE. This model leverages temporal adaptive dynamic stock relationship graphs, facilitating improved information transfer among stocks within the dynamic probabilistic factor model. Experimental results on three real stock market datasets demonstrate that our method outperforms various state-of-the-art approaches.
GameTrail: Probabilistic Lifecycle Process Model for Deep Game Understanding
Authors: Shanyang Jiang, Lan Zhang, Hui Xu, Jiahui Huang, Qi He, Xing Zhou, Lei Huang, Jie Jiang
Venue: CIKM
Year: 2024

As the mobile gaming market experiences significant growth, there is a continuous emergence of new gaming products such as premium games and video mini-games. Meeting their marketing needs and supporting their business growth is essential for long-term prosperity. However, compared with extensive studies on user modeling such as LTV prediction, much less attention has been drawn to special gaming products, especially in terms of understanding their lifecycle stages and corresponding demands. Unlike modeling individual users, understanding games is closely tied to user behavior: the lifecycle of a game encompasses the entire process from initial user interaction to churn, and by accurately identifying and tracking the evolution of the game lifecycle can lead to better personal service. This raises the necessity of comprehensively understanding the lifecycle process model of the game. In this paper, we introduce the GameTrail - Probabilistic Lifecycle Process Model, designed to construct the complete lifecycle and stage representation for games and users through long-term repeated interactions. Specifically, we first initiate the complete game lifecycle using a joint probabilistic stochastic process model by defining the lifecycle stages of both games and users as latent variables and learning it via Bayesian Variation Inference. Furthermore, we employ cross attention and online embedding learning to capture the more recent advertising context changes in the in-game stage transitions. Finally, we collect various games' data from public resources to construct an experimental dataset for our experiments. Meanwhile, more comprehensive experiments conducted on real-world gaming industry datasets have showcased the effectiveness of our approach, showing a relative improvement of 48% and 17% on NMSE and NMAE than the live baseline.
Tackling Noisy Clients in Federated Learning with End-to-end Label Correction
Authors: Xuefeng Jiang, Sheng Sun, Jia Li, Jingjing Xue, Runhan Li, Zhiyuan Wu, Gang Xu, Yuwei Wang, Min Liu
Venue: CIKM
Year: 2024

Recently, federated learning (FL) has achieved wide successes for diverse privacy-sensitive applications without sacrificing the sensitive private information of clients. However, the data quality of client datasets can not be guaranteed since corresponding annotations of different clients often contain complex label noise of varying degrees, which inevitably causes the performance degradation. Intuitively, the performance degradation is dominated by clients with higher noise rates since their trained models contain more misinformation from data, thus it is necessary to devise an effective optimization scheme to mitigate the negative impacts of these noisy clients. In this work, we propose a two-stage framework FedELC to tackle this complicated label noise issue. The first stage aims to guide the detection of noisy clients with higher label noise, while the second stage aims to correct the labels of noisy clients' data via an end-to-end label correction framework which is achieved by learning possible ground-truth labels of noisy clients' datasets via back propagation. We implement sixteen related methods and evaluate five datasets with three types of complicated label noise scenarios for a comprehensive comparison. Extensive experimental results demonstrate our proposed framework achieves superior performance than its counterparts for different scenarios. Additionally, we effectively improve the data quality of detected noisy clients' local datasets with our label correction framework. The code is available at https://github.com/Sprinter1999/FedELC.
Deep Uncertainty-Based Explore for Index Construction and Retrieval in Recommendation System
Authors: Xin Jiang, Kaiqiang Wang, Yinlong Wang, Fengchang Lv, Taiyang Peng, Shuai Yang, Xianteng Wu, Pengye Zhang, Shuo Yuan, Yifan Zeng
Venue: CIKM
Year: 2024

In recommendation systems, the relevance and novelty of the final results are selected through a cascade system of Matching -> Ranking -> Strategy. The matching model serves as the starting point of the pipeline and determines the upper bound of the subsequent stages. Balancing the relevance and novelty of matching results is a crucial step in the design and optimization of recommendation systems, contributing significantly to improving recommendation quality. However, the typical matching algorithms have not simultaneously addressed the relevance and novelty perfectly. One main reason is that deep matching algorithms exhibit significant uncertainty when estimating items in the long tail (e.g., due to insufficient training samples) items. The uncertainty not only affects the training of the models but also influences the confidence in the index construction and beam search retrieval process of these models. This paper proposes the UICR (Uncertainty-based explore for Index Construction and Retrieval) algorithm, which introduces the concept of uncertainty modeling in the matching stage and achieves multi-task modeling of model uncertainty and index uncertainty. The final matching results are obtained by combining the relevance score and uncertainty score infered by the model. Experimental results demonstrate that the UICR improves novelty without sacrificing relevance on real-world industrial productive environments and multiple open-source datasets. Remarkably, online A/B test results of display advertising in Shopee demonstrates the effectiveness of the proposed algorithm.
A Multi-Node Multi-GPU Distributed GNN Training Framework for Large-Scale Online Advertising
Authors: Xuewu Jiao, Xinsheng Luo, Miao Li, Jiang Bian, Junchao Yang, Wei Hu, Mingqing Hu, Weipeng Lu, Shikun Feng, Danlei Feng, Dongxu Yang, Haoyi Xiong, Shuanglong Li, Lin Liu
Venue: CIKM
Year: 2024

Graph Neural Networks (GNNs) have become critical in various domains such as online advertising but face scalability challenges due to the growing size of graph data, leading to the needs for advanced distributed GPU computation strategies across multiple nodes. This paper presents PGLBox-Cluster, a robust distributed graph learning framework constructed atop the PaddlePaddle platform, implemented to efficiently process graphs comprising billions of nodes and edges. Through strategic partitioning of the model, node attributes, and graph data and leveraging industrial-grade RPC and NCCL for communication, PGLBox-Cluster facilitates effective distributed computation. The extensive experimental results confirm that PGLBox-Cluster achieves a 1.94x to 2.93x speedup over the single-node configuration, significantly elevating graph neural network scalability and efficiency by handling datasets exceeding 3 billion nodes and 120 billion edges with its novel asynchronous communication and graph partitioning techniques. The repository is released at This Link.
CNN to GNN: Unsupervised Multi-level Knowledge Learning
Authors: Ziheng Jiao, Hongyuan Zhang, Xuelong Li
Venue: CIKM
Year: 2024

Although graph neural networks (GNNs) can extract the latent relationship-level knowledge among the graph nodes and have achieved excellent performance in unsupervised scenarios, it is weak in learning the instance-level knowledge in contrast to the convolution neural networks (CNNs). Besides, lacking of the graph structure limits the extension of GNNs on non-graph datasets. To solve these problems, we propose a novel unsupervised multi-level knowledge fusion network. It successfully unifies the instance-level and relationship-level knowledge on the non-graph data by distillation from a pre-trained CNN teacher to a GNN student. Meanwhile, a sparse weighted strategy is designed to adaptively extract the sparse graph topology and extend the GNN on non-graph datasets. By optimization of distillation loss, the "boosted'' GNN student can learn the multi-level knowledge and extract more discriminative deep embeddings for clustering. Finally, extensive experiments show it has achieved excellent performance compared with the current methods.
Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation
Authors: Baoyu Jing, Dawei Zhou, Kan Ren, Carl Yang
Venue: CIKM
Year: 2024

Spatiotemporal time series are usually collected via monitoring sensors placed at different locations, which usually contain missing values due to various failures, such as mechanical damages and Internet outages. Imputing the missing values is crucial for analyzing time series. When recovering a specific data point, most existing methods consider all the information relevant to that point regardless of the cause-and-effect relationship. During data collection, it is inevitable that some unknown confounders are included, e.g., background noise in time series and non-causal shortcut edges in the constructed sensor network. These confounders could open backdoor paths and establish non-causal correlations between the input and output. Over-exploiting these non-causal correlations could cause overfitting. In this paper, we first revisit spatiotemporal time series imputation from a causal perspective and show how to block the confounders via the frontdoor adjustment. Based on the results of frontdoor adjustment, we introduce a novel Causality-Aware Spatiotemporal Graph Neural Network (Casper), which contains a novel Prompt Based Decoder (PBD) and a Spatiotemporal Causal Attention (SCA). PBD could reduce the impact of confounders and SCA could discover the sparse causal relationships among embeddings. Theoretical analysis reveals that SCA discovers causal relationships based on the values of gradients. We evaluate Casper on three real-world datasets, and the experimental results show that Casper could outperform the baselines and could effectively discover the causal relationships.
Automated Contrastive Learning Strategy Search for Time Series
Authors: Baoyu Jing, Yansen Wang, Guoxin Sui, Jing Hong, Jingrui He, Yuqing Yang, Dongsheng Li, Kan Ren
Venue: CIKM
Year: 2024

In recent years, Contrastive Learning (CL) has become a predominant representation learning paradigm for time series. Most existing methods manually build specific CL Strategies (CLS) by human heuristics for certain datasets and tasks. However, manually developing CLS usually requires excessive prior knowledge about the data, and massive experiments to determine the detailed CL configurations. In this paper, we present an Automated Machine Learning (AutoML) practice at Microsoft, which automatically learns CLS for time series datasets and tasks, namely Automated Contrastive Learning (AutoCL). We first construct a principled search space of size over 3 × 1012, covering data augmentation, embedding transformation, contrastive pair construction, and contrastive losses. Further, we introduce an efficient reinforcement learning algorithm, which optimizes CLS from the performance on the validation tasks, to obtain effective CLS within the space. Experimental results on various real-world datasets demonstrate that AutoCL could automatically find the suitable CLS for the given dataset and task. From the candidate CLS found by AutoCL on several public datasets/tasks, we compose a transferable Generally Good Strategy (GGS), which has a strong performance for other datasets. We also provide empirical analysis as a guide for the future design of CLS.
Introducing CausalBench: A Flexible Benchmark Framework for Causal Analysis and Machine Learning
Authors: Ahmet Kapkiç, Pratanu Mandal, Shu Wan, Paras Sheth, Abhinav Gorantla, Yoonhyuk Choi, Huan Liu, K. Selçuk Candan
Venue: CIKM
Year: 2024

While witnessing the exceptional success of machine learning (ML) technologies in many applications, users are starting to notice a critical shortcoming of ML: correlation is a poor substitute for causation. The conventional way to discover causal relationships is to use randomized controlled experiments (RCT); in many situations, however, these are impractical or sometimes unethical. Causal learning from observational data offers a promising alternative. While being relatively recent, causal learning aims to go far beyond conventional machine learning, yet several major challenges remain. Unfortunately, advances are hampered due to the lack of unified benchmark datasets, algorithms, metrics, and evaluation service interfaces for causal learning. In this paper, we introduce CausalBench, a transparent, fair, and easy-to-use evaluation platform, aiming to (a) enable the advancement of research in causal learning by facilitating scientific collaboration in novel algorithms, datasets, and metrics and (b) promote scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench provides services for benchmarking data, algorithms, models, and metrics, impacting the needs of a broad of scientific and engineering disciplines.
DirDense: A Tool for Mining Dense Subgraphs from a Big Directed Graph
Authors: Jalal Khalil, Akhlaque Ahmad, Da Yan, Lyuheng Yuan, Saugat Adhikari, Yang Zhou, Zhe Jiang
Venue: CIKM
Year: 2024

Mining dense subgraphs from a big graph is important in applications such as community (or module) detection in social (or biological) networks. While most dense structures are defined on undirected graphs, recent efforts have generalized these notions to directed graphs. In this demonstration paper, we present DirDense, an interactive tool that makes it easy for end-users to mine dense structures from a big directed graph. DirDense currently supports the mining of maximal (γ1, γ2)-quasi-cliques, maximal (k 1,k 1)-plexes, and the directed densest subgraph. DirDense facilitates parameter tuning for each type of the structure-mining tasks, and provides intuitive interfaces to visualize and examine the dense directed structures. Using real-world data, we showcase how users can mine dense directed structures by parameter tuning in DirDense, and how they can conveniently examine these structures and cascade the mining tasks to find progressively larger dense subgraphs more quickly.
XCapsUTL: Cross-domain Unsupervised Transfer Learning Framework using a Capsule Neural Network
Authors: Naman Khetan, Sanyog Dewani, Gokul Swamy, Vikalp Gajbhiye
Venue: CIKM
Year: 2024

As e-commerce stores broaden their reach into new regions and introduce new products within established markets, the development of effective machine learning models becomes increasingly challenging due to the scarcity of labeled data. Traditional transfer learning methods typically require some labeled data from the target domain and often face computational bottlenecks. Despite the availability of a few transfer learning techniques, most are primarily developed for vision and text applications, making them unsuitable for other types of data. In many industries, however, tabular data is a predominant and crucial data type. Our work introduces XCapsUTL, a novel unsupervised transfer learning framework specifically designed for tabular data, aiming to fill this significant gap. Our approach leverages Capsule Neural Networks (CapsNet) to extract domain-agnostic knowledge. This knowledge is then refined using a constrained fine-tuning process, ensuring adaptability to the target task while preserving learned representations. XCapsUTL's unique feature encapsulation capabilities within CapsNet promote effective knowledge transfer without the need for designing effective feature-wise interaction approaches to capture higher-level semantics. Extensive experiments demonstrate the robustness and generalization capabilities of XCapsUTL across multiple domains and datasets, highlighting its practical significance and utility in addressing the unique challenges of tabular data in industry settings.
Flexi-clique: Exploring Flexible and Sub-linear Clique Structures
Authors: Song Kim, Junghoon Kim, Susik Yoon, Jungeun Kim
Venue: CIKM
Year: 2024

Identifying cohesive subgraphs within networks is a fundamental problem in graph theory, relevant to various domains. The traditional clique problem, which finds fully connected subgraphs, often faces limitations due to its strict connectivity requirements. This paper introduces a novel degree-based relaxation model called Flexi-clique, where the degree constraint is adjusted sub-linearly based on the subgraph size. We establish that the maximum Flexi-clique problem is NP-hard and propose an efficient and effective peeling algorithm to address it. Our extensive experimental evaluation of real-world datasets demonstrates the effectiveness and efficiency of our approach in discovering large, cohesive subgraphs in networks.