OpenRaiser/Intern-Atlas huggingface dataset:
CLOCK: Online Temporal Hierarchical Framework for Multi-scale Multi-granularity Forecasting of User Impression
Authors: XiaoYu Wang, Yonghui Guo, Xiaoyang Ma, Dongbo Huang, Lan Xu, Haisheng Tan, Hao Zhou, Xiang-Yang Li
Venue: CIKM
Year: 2023

User impression forecasting underpins various commercial activities, from long-term strategic decisions to short-term automated operations. As a representative that involves both kinds, the highly profitable Guaranteed Delivery (GD) advertising focuses mainly on promoting brand effect by allowing advertisers to order target impressions weeksin advance and get allocatedonline at the scheduled time. Such a business mode naturally incurs three issues making existing solutions inferior: 1) Timescale-granularity dilemma of coherently supporting the sales of day-level impressions of the distant future and the corresponding fine-grained allocation in real-time. 2) High dimensionality due to the Cartesian product of user attribute combinations. 3) Stability-plasticity dilemma of instant adaptation to emerging patterns of temporal dependency withoutcatastrophic forgetting of repeated ones facing the non-stationary traffic. To overcome the obstacles, we propose an online temporal hierarchical framework that functions analogously to a CLOCK and hence its name. Long-timescale, coarse-grained temporal data (e.g., the daily impression of one quarter) and short-timescale but fine-grained ones are handled separately by dedicated models, just like the hour/minute/second hands. Each tier in the hierarchy is triggered for forecasting and updating by need at different frequencies, thus saving the maintenance overhead. Furthermore, we devise a reconciliation mechanism to coordinate tiers by aggregating the separately learned local variance and global trends tier by tier. CLOCK solves the dimensionality dilemma by subsuming the autoencoder design to achieve an end-to-end, nonlinear factorization of streaming data into a low-dimension latent space, where a neural predictor produces predictions for the decoder to project them back to the high dimension. Lastly, we regulate the CLOCK's continual refinement by combining the complementary Experience Replay (ER) and Knowledge Distillation (KD) techniques to consolidate and recall previously learned temporal patterns. We conduct extensive evaluations on three public datasets and the real-life user impression log from the Tencent advertising system, and the results demonstrate CLOCK's efficacy.
Training Heterogeneous Graph Neural Networks using Bandit Sampling
Authors: Ta-Yang Wang, Rajgopal Kannan, Viktor K. Prasanna
Venue: CIKM
Year: 2023

Graph neural networks (GNNs) have gained significant attention across diverse areas due to their superior performance in learning graph representations. While GNNs exhibit superior performance compared to other methods, they are primarily designed for homogeneous graphs, where all nodes and edges are of the same type. Training a GNN model for large-scale graphs incurs high computation and storage costs, especially when considering the heterogeneous structural information of each node. To address the demand for efficient GNN training, various sampling methods have been proposed. In this paper, we propose a sampling method based on bandit sampling, an online learning algorithm with provable convergence under weak assumptions on the learning objective. To the best of our knowledge, this is the first bandit-based sampling method applied to heterogeneous GNNs with a theoretical guarantee. The main idea is to prioritize node types with more informative connections with respect to the learning objective. Compared with existing techniques for GNN training on heterogeneous graphs, extensive experiments using the Open Academic Graph (OAG) dataset demonstrate that our proposed method outperforms the state-of-the-art in terms of the runtime across various tasks with a speed-up of 1.5-2x, while achieving similar accuracy.
Network Embedding with Adaptive Multi-hop Contrast
Authors: Chenhao Wang, Yong Liu, Yan Yang
Venue: CIKM
Year: 2023

\beginabstract Graph neural networks (GNNs) have shown strong performance in graph-based analysis tasks. Despite their remarkable success, the inherent homophilic message-passing mechanism (MP) makes GNNs challenging to generalize to heterophilic graphs. In addition, the MP explicitly exploits the connection relationships between local neighbor nodes making GNNs unable to maintain stable performance in the face of adversarial perturbation attacks. In this paper, we propose a new method to explore graph structure by removing explicit message-passing mechanisms and present a network embedding framework AMCNE with Adaptive Multi-hop Contrast loss (AMCLoss) to address these challenges. AMCNE only relies on a simple autoencoder to obtain node representations for classification and uses elaborate contrastive loss to drive nodes capturing complex structural information on heterophilic graphs. The comprehensive experiments show that AMCNE outperforms state-of-the-art baseline models on homophilic and heterophilic graphs and is more robust in the node classification task. \endabstract
The 1st Workshop on Recommendation with Generative Models
Authors: Wenjie Wang, Yong Liu, Yang Zhang, Weiwen Liu, Fuli Feng, Xiangnan He, Aixin Sun
Venue: CIKM
Year: 2023

The boom of generative models has paved the way for significant advances in recommender systems. For instance, pre-trained generative models offer unprecedented opportunities to improve recommender algorithms for user modeling. This workshop aims to provide a platform for researchers to actively explore and share innovative ideas on integrating generative models into recommender systems, mainly focusing on five key aspects: (i) enhancing recommender algorithms, (ii) generating personalized content in some scenarios such as micro-videos, (iii) changes in the user-system interaction paradigm, (iv) boosting trustworthiness checks, and (v) evaluation methodologies of generative recommendation. With the rapid development of generative models, a growing number of studies along the above directions are emerging, revealing the timeliness and necessity of this workshop. The related research will bring novel features to recommender systems and contribute to new tasks and technologies in both academia and industry. In the long run, this research direction might revolutionize the traditional recommender paradigm and lead to the maturation of next-generation recommender systems.
Optimizing Upstream Representations for Out-of-Domain Detection with Supervised Contrastive Learning
Authors: Bo Wang, Tsunenori Mine
Venue: CIKM
Year: 2023

Out-of-Domain (OOD) text detection has attracted significant research interest. However, conventional approaches primarily employ Cross-Entropy loss during upstream encoder training and seldom focus on optimizing discriminative In-Domain (IND) and OOD representations. To fill this gap, we introduce a novel method that applies supervised contrastive learning (SCL) to IND data for upstream representation optimization. This effectively brings the embeddings of semantically similar texts together while pushing dissimilar ones further apart, leading to more compact and distinct IND representations. This optimization subsequently improves the differentiation between IND and OOD representations, thereby enhancing the detection effect in downstream tasks. To further strengthen the ability of SCL to consolidate IND embedding clusters, and to improve the generalizability of the encoder, we propose a method that generates two different variations of the same text as "views". This is achieved by applying a twice "dropped-out" on the embeddings before performing SCL. Extensive experiments indicate that our method not only outperforms state-of-the-art approaches, but also reduces the requirement for training a large 354M-parameter model down to a more efficient 110M-parameter model, highlighting its superiority in both effectiveness and computational economy.
Diversity-aware Deep Ranking Network for Recommendation
Authors: Zihong Wang, Yingxia Shao, Jiyuan He, Jinbao Liu, Shitao Xiao, Tao Feng, Ming Liu
Venue: CIKM
Year: 2023

Diversity is a vital factor in recommendation systems.Improving the diversity in recommendations helps broaden users' horizons, bring good user experience and promote the enterprises' sales. In the past years, many efforts have been devoted to optimizing the diversity in the matching stage and the re-ranking stage of the recommendation system, but few in the ranking stage. The ranking stage is the intermediate stage of the recommendation system. Improving the diversity of the ranking stage can preserve the diversity of the matching stage, and provide a more diversified list for the re-ranking stage. Besides, the ranking models are able to achieve a better balance between accuracy and diversity. In this paper, we aim to improve the diversity in the ranking stage. To address the diversity challenges posed by the pointwise ranking model and biased user interaction history, we propose a Diversity-aware Deep Ranking Network by carefully designing two diversity-aware components that are diversity-aware listwise information fusion and balanced weighting loss. We conduct both offline and online experiments, and the results demonstrate that our proposed model effectively improves the recommendation diversity in the ranking stage while maintaining the accuracy. Moreover, the new model achieves 1.27%, 2.30% and 1.98% improvements in VBR, GMV and Coverage in Meituan, one of the world's largest E-commerce platforms.
Clustering-property Matters: A Cluster-aware Network for Large Scale Multivariate Time Series Forecasting
Authors: Yuan Wang, Zezhi Shao, Tao Sun, Chengqing Yu, Yongjun Xu, Fei Wang
Venue: CIKM
Year: 2023

Large-scale Multivariate Time Series(MTS) widely exist in various real-world systems, imposing significant demands on model efficiency. A recent work, STID, addressed the high complexity issue of popular Spatial-Temporal Graph Neural Networks(STGNNs). Despite its success, when applied to large-scale MTS data, the number of parameters of STID for modeling spatial dependencies increases substantially, leading to over-parameterization issues and suboptimal performance. These observations motivate us to explore new approaches for modeling spatial dependencies in a parameter-friendly manner. In this paper, we argue that the spatial properties of variables are essentially the superposition of multiple cluster centers. Accordingly, we propose a Cluster-Aware Network(CANet), which effectively captures spatial dependencies by mining the implicit cluster centers of variables. CANet solely optimizes the cluster centers instead of the spatial information of all nodes, thereby significantly reducing the parameter amount. Extensive experiments on two large-scale datasets validate our motivation and demonstrate the superiority of CANet.
Efficient Multi-Task Learning via Generalist Recommender
Authors: Luyang Wang, Cangcheng Tang, Chongyang Zhang, Jun Ruan, Kai Huang, Jason Jinquan Dai
Venue: CIKM
Year: 2023

Multi-task learning (MTL) is a common machine learning technique that allows the model to share information across different tasks and improve the accuracy of recommendations for all of them. Many existing MTL implementations suffer from scalability issues as the training and inference performance can degrade with the increasing number of tasks, which can limit production use case scenarios for MTL-based recommender systems. Inspired by the recent advances of large language models, we developed an end-to-end efficient and scalable Generalist Recommender (GRec). GRec takes comprehensive data signals by utilizing NLP heads, parallel Transformers, as well as a wide and deep structure to process multi-modal inputs. These inputs are then combined and fed through a newly proposed task-sentence level routing mechanism to scale the model capabilities on multiple tasks without compromising performance. Offline evaluations and online experiments show that GRec significantly outperforms our previous recommender solutions. GRec has been successfully deployed on one of the largest telecom websites and apps, effectively managing high volumes of online traffic every day.
FAMC-Net: Frequency Domain Parity Correction Attention and Multi-Scale Dilated Convolution for Time Series Forecasting
Authors: Min Wang, Hua Wang, Fan Zhang
Venue: CIKM
Year: 2023

In recent years, time series forecasting models based on the Transformer framework have shown great potential, but they suffer from the inherent drawback of high computational complexity and only focus on global modeling. Inspired by trend-seasonality decomposition, we propose a method that combines global modeling with local feature extraction within the seasonal cycle. It aims at capturing the global view while fully exploring the potential features within each seasonal cycle and better expressing the long-term and periodic characteristics of time series. We introduce a frequency domain parity correction block to compute global attention and utilize multi-scale dilated convolution to extract local correlations within each cycle. Additionally, we adopt a dual-branch structure to separately model the seasonality and trend based on their intrinsic features, improving prediction performance and enhancing model interpretability. This model is implemented on a completely single-layer decoder architecture, breaking through the traditional encoder-decoder architecture paradigm and reducing computational complexity to a certain extent. We conducted sufficient experimental validation on eight benchmark datasets, and the results demonstrate its superior performance compared to existing methods in both univariate and multivariate forecasting.
MPerformer: An SE(3) Transformer-based Molecular Perceptron
Authors: Fanmeng Wang, Hongteng Xu, Xi Chen, Shuqi Lu, Yuqing Deng, Wenbing Huang
Venue: CIKM
Year: 2023

Molecular perception aims to construct 3D molecules from 3D atom clouds (i.e., atom types and corresponding 3D coordinates), determining bond connections, bond orders, and other molecular attributes within molecules. It is essential for realizing many applications in cheminformatics and bioinformatics, such as modeling quantum chemistry-derived molecular structures in protein-ligand complexes. Additionally, many molecular generation methods can only generate molecular 3D atom clouds, requiring molecular perception as a necessary post-processing. However, existing molecular perception methods mainly rely on predefined chemical rules and fail to leverage 3D geometric information, whose performance is sub-optimal fully. In this study, we propose MPerformer, an SE(3) Transformer-based molecular perceptron exhibiting SE(3)-invariance, to construct 3D molecules from 3D atom clouds efficiently. Besides, we propose a multi-task pretraining-and-finetuning paradigm to learn this model. In the pretraining phase, we jointly minimize an attribute prediction loss and an atom cloud reconstruction loss, mitigating the data imbalance issue of molecular attributes and enhancing the robustness and generalizability of the model. Experiments show that MPerformer significantly outperforms state-of-the-art molecular perception methods in precision and robustness, benefiting various molecular generation scenarios.
AFRF: Angle Feature Retrieval Based Popularity Forecasting
Authors: Haoyu Wang, Zongxia Xie, Meiyao Liu, Canhua Guan
Venue: CIKM
Year: 2023

Social media popularity forecasting has become a hot research topic in recent years. It is of great significance in assisting public opinion monitoring and advertising placement. Time series prediction is one of the simple and commonly used methods for popularity forecasting, which takes the popularity of the first few time steps in the observed data as inputs. However, the complete popularity trend of each social media is known in the training dataset, while the historical time series information except for the first few time steps is neglected in the existing models. In order to utilize the complete historical information from the observed data, a retrieval method is introduced in this paper. Therefore, how to retrieve similar social media based on the first few steps time series and how to integrate the similar historical information have become two challenges. A two-stage prediction method named Angle Feature Retrieval based Forecasting (AFRF) is proposed in this paper to solve the upper two problems. In the first stage, based on the angle features of series, we retrieve K similar series from the historical posts and concatenate them with the target series as the model's input. In the second stage, an attention mechanism is used to learn the temporal relationships among the series and generate future popularity forecasts. We evaluated the multi-step and single-point forecasting performance of AFRF on three real-world datasets and compared it with state-of-the-art popularity forecasting methods, such as temporal feature-based and cascade-based methods, verifying the effectiveness of AFRF.
UrbanFloodKG: An Urban Flood Knowledge Graph System for Risk Assessment
Authors: Yu Wang, Feng Ye, Binquan Li, Gaoyang Jin, Dong Xu, Fengsheng Li
Venue: CIKM
Year: 2023

Increasing numbers of people live in flood-prone areas worldwide. With continued development, urban flood will become more frequent, which has caused casualties and property damage. Researchers have been dedicating to urban flood risk assessments in recent years. However, current research is still facing the challenges of multi-modal data fusion and knowledge representation of urban flood events. Therefore, in this paper, we propose an Urban Flood Knowledge Graph (UrbanFloodKG) system that enables KG to support urban flood risk assessment. The system consists of data layer, graph layer, algorithm layer, and application layer, which implements knowledge extraction and storage functions, integrates knowledge representation learning models and graph neural network models to support link prediction and node classification tasks. We conduct model comparison experiments on link prediction and node classification tasks based on urban flood event data from Guangzhou, and demonstrate the effectiveness of the models used. Our experiments prove that the accuracy of risk assessment can reach 91% when using GEN, which provides a a promising research direction for urban flood risk assessment.
MultiPLe: Multilingual Prompt Learning for Relieving Semantic Confusions in Few-shot Event Detection
Authors: Siyuan Wang, Jianming Zheng, Wanyu Chen, Fei Cai, Xueshan Luo
Venue: CIKM
Year: 2023

Event detection (ED) is a challenging task in the field of information extraction. Due to the monolingual text and rampant confusing triggers, traditional ED models suffer from semantic confusions in terms of polysemy and synonym, leading to severe detection mistakes. Such semantic confusions can be further exacerbated in a practical situation where scarce labeled data cannot provide sufficient semantic clues. To mitigate such bottleneck, we propose a multilingual prompt learning (MultiPLe) framework for few-shot event detection (FSED), including three components, i.e., a multilingual prompt, a hierarchical prototype and a quadruplet contrastive learning module. In detail, to ease the polysemy confusion, the multilingual prompt module develops the in-context semantics of triggers via the multilingual disambiguation and prior knowledge in pretrained language models. Then, the hierarchical prototype module is adopted to diminish the synonym confusion by connecting the captured inmost semantics of fuzzy triggers with labels at a fine granularity. Finally, we employ the quadruplet contrastive learning module to tackle the insufficient label representation and potential noise. Experiments on two public datasets show that MultiPLe outperforms the state-of-the-art baselines in weighted F1-score, presenting a maximum improvement of 13.63% for FSED.
Node-dependent Semantic Search over Heterogeneous Graph Neural Networks
Authors: Zhenyi Wang, Huan Zhao, Fengqi Liang, Chuan Shi
Venue: CIKM
Year: 2023

In recent years, Heterogeneous Graph Neural Networks (HGNNs) have been the state-of-the-art approaches for various tasks on Heterogeneous Graphs (HGs), e.g., recommendation and social network analysis. Despite the success of existing HGNNs, the utilization of the intricate semantic information in HGs is still insufficient. In this work, we study the problem of how to design powerful HGNNs under the guidance of node-dependent semantics. Specifically, to perform semantic search over HGNNs, we propose to develop semantic structures in terms of relation selection and connection selection, which could guide a task-relevant message flow. Furthermore, to better capture the diversified property of different node samples in HGs, we design predictors to adaptively decide the semantic structures per node. Extensive experiments on seven benchmarking datasets across different downstream tasks, i.e., node classification and recommendation, show that our method can consistently outperform various state-of-the-art baselines with shorter inference latency, which justifies its effectiveness and efficiency. The code and data are available at https://github.com/BUPT-GAMMA/NDS.
Modeling Preference as Weighted Distribution over Functions for User Cold-start Recommendation
Authors: Jingxuan Wen, Huafeng Liu, Liping Jing
Venue: CIKM
Year: 2023

=User cold-start recommendation is a well-known challenge in current recommender systems. The cause is that the number of user interactions is too few to accurately estimate user preferences. Furthermore, the uncertainty of user interactions intensifies along with the number of user interactions decreasing. Although existing meta-learning based models with globally sharing knowledge show good performance in most cold-start scenarios, the ability of handling challenges on intention importance and prediction uncertainty is missing: (1) Intra-user uncertainty. When estimating user preferences (reflected in the user's latent representation), each of user interactions is independently considered in the form of user-item pair, which cannot capture the correlation between user interactions, as well as considering the global intent under user interactions. (2) Inter-user importance. During the model training, all users are treated as equally important, which cannot distinguish the contribution of users in the model training process. Assigning the same weight to all users may lead to users with high uncertainty incorrectly guiding the model learning in the early stage of training. To tackle the above challenges, in this paper, we focus on modeling user preference as a weighted distribution over functions (WDoF) for user cold-start recommendation, which not only models the intra-user uncertainty through neural processes with Multinomial likelihood but also considers the importance of different users with curriculum learning during the model training process. Furthermore, we provide a theoretical explanation that why the proposed model performs better than regular neural processes based recommendation methods. Experiments on four real-world datasets demonstrate the effectiveness of the proposed model over several state-of-the-art cold-start recommendation methods.