OpenRaiser/Intern-Atlas huggingface dataset:
Multitask Ranking System for Immersive Feed and No More Clicks: A Case Study of Short-Form Video Recommendation
Authors: Qingyun Liu, Zhe Zhao, Liang Liu, Zhen Zhang, Junjie Shan, Yuening Li, Shuchao Bi, Lichan Hong, Ed H. Chi
Venue: CIKM
Year: 2023

In recent years, social media users spend significant amount of time on Short-Form Video (SFV) platforms. Its success in creating an immersive viewership experience is not only from the content, but also due to its unique UI innovation: instead of providing choices for users to click, SFV platforms actively recommend content to users to watch one at a time. In this paper, we highlight unique challenges rooted from such UI changes for SFV recommendation system design. Firstly, there is yet much unexplored for sources of system biases under the new UI, as there are no clicks nor the common click-based position biases. Additionally, when training multiple types of user activities, positive labels for activities like sharing and commenting can be much sparser and more skewed than traditional click-based recommendation systems, as the latter can filter non-click impressions when generating "post-click" activities. To tackle these challenges, we introduce a unified multi-task ranking framework which puts two novel components all together into an overall system for SFV recommendation. First, we identify that there are position biases of SFVs in the recommendation sequence, namely "watch trail biases", and introduce biases correction using trail-related information. Second, to get the most benefits from multi-task learning, especially co-training tasks with extremely skewed and sparse labels, we adapt a disentangle regularization to mitigate task conflicts, introduce loss upweighting for sparse task co-training and adopt a meta-learning algorithm for efficient weight selection. We demonstrate the effectiveness and efficiency of the framework on one of today's largest SFV platforms. Our framework has been deployed to the production system for more than 6 months.
Hierarchical Prompt Tuning for Few-Shot Multi-Task Learning
Authors: Jingping Liu, Tao Chen, Zujie Liang, Haiyun Jiang, Yanghua Xiao, Feng Wei, Yuxi Qian, Zhenghong Hao, Bing Han
Venue: CIKM
Year: 2023

Prompt tuning has enhanced the performance of Pre-trained Language Models for multi-task learning in few-shot scenarios. However, existing studies fail to consider that the prompts among different layers in Transformer are different due to the diverse information learned at each layer. In general, the bottom layers in the model tend to capture low-level semantic or structural information, while the upper layers primarily acquire task-specific knowledge. Hence, we propose a novel hierarchical prompt tuning model for few-shot multi-task learning to capture this regularity. The designed model mainly consists of three types of prompts: shared prompts, auto-adaptive prompts, and task-specific prompts. Shared prompts facilitate the sharing of general information across all tasks. Auto-adaptive prompts dynamically select and integrate relevant prompt information from all tasks into the current task. Task-specific prompts concentrate on learning task-specific knowledge. To enhance the model's adaptability to diverse inputs, we introduce deep instance-aware language prompts as the foundation for constructing the above prompts. To evaluate the effectiveness of our proposed method, we conduct extensive experiments on multiple widely-used datasets. The experimental results demonstrate that the proposed method achieves state-of-the-art performance for multi-task learning in few-shot settings and outperforms ChatGPT in the full-data setting.
Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting
Authors: Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, Xuan Song
Venue: CIKM
Year: 2023

With the rapid development of the Intelligent Transportation System (ITS), accurate traffic forecasting has emerged as a critical challenge. The key bottleneck lies in capturing the intricate spatio-temporal traffic patterns. In recent years, numerous neural networks with complicated architectures have been proposed to address this issue. However, the advancements in network architectures have encountered diminishing performance gains. In this study, we present a novel component called spatio-temporal adaptive embedding that can yield outstanding results with vanilla transformers. Our proposed Spatio-Temporal Adaptive Embedding transformer (STAEformer) achieves state-of-the-art performance on five real-world traffic forecasting datasets. Further experiments demonstrate that spatio-temporal adaptive embedding plays a crucial role in traffic forecasting by effectively capturing intrinsic spatio-temporal relations and chronological information in traffic time series.
UniTE: A Unified Treatment Effect Estimation Method for One-sided and Two-sided Marketing
Authors: Runshi Liu, Zhipeng Hou
Venue: CIKM
Year: 2023

Many internet platforms are two-sided markets that involve two types of participants. Examples include e-commerce platforms like Taobao (retailers and consumers) and ride-hailing platforms like Uber (drivers and passengers). Participants of different types in the two-sided market have relationships (i.e., supply and demand) that provide externalities and network benefits. On two-sided platforms, marketing campaigns are designed by subsidizing supply or demand. Uplift models built in this scenario usually consider the treatment assignment for only one of the two sides. However, ignoring the interaction of treatments between two sides or treating them as noises may result in incomplete models and inaccurate predictions. As far as we know, there is not much work related to modeling the combinational treatment effects in the two-sided market. In this paper, we first introduce the two-sided treatment effects estimation problem and then propose a Unified Treatment effect Estimation (UniTE) method for one-sided and two-sided marketing. We extend the Robinson Decomposition to two-sided, in which the relationship of the three involved tasks, namely the outcome, the propensity, and the treatment effect, is theoretically derived. Based on the decomposition result, a multi-task-based neural network model is proposed to integrate the three tasks and learn the inter-task-related common information, which prompts the model to estimate the treatment effects better. We also propose a unified synthetic data generation method that adapts to one/two-sided situations to verify the treatment effects estimation performance. Extensive and comprehensive experimental results show that our method outperforms the other methods.
ForeSeer: Product Aspect Forecasting Using Temporal Graph Embedding
Authors: Zixuan Liu, Gaurush Hiranandani, Kun Qian, Edward W. Huang, Yi Xu, Belinda Zeng, Karthik Subbian, Sheng Wang
Venue: CIKM
Year: 2023

Developing text mining approaches to mine aspects from customer reviews has been well-studied due to its importance in understanding customer needs and product attributes. In contrast, it remains unclear how to predict the future emerging aspects of a new product that currently has little review information. This task, which we named product aspect forecasting, is critical for recommending new products, but also challenging because of the missing reviews. Here, we propose ForeSeer, a novel textual mining and product embedding approach progressively trained on temporal product graphs for this novel product aspect forecasting task. ForeSeer transfers reviews from similar products on a large product graph and exploits these reviews to predict aspects that might emerge in future reviews. A key novelty of our method is to jointly provide review, product, and aspect embeddings that are both time-sensitive and less affected by extremely imbalanced aspect frequencies. We evaluated ForeSeer on a real-world product review system containing 11,536,382 reviews and 11,000 products over 3 years. We observe that ForeSeer substantially outperformed existing approaches with at least 49.1% AUPRC improvement under the real setting where aspect associations are not given. ForeSeer further improves future link prediction on the product graph and the review aspect association prediction. Collectively, Foreseer offers a novel framework for review forecasting by effectively integrating review text, product network, and temporal information, opening up new avenues for online shopping recommendation and e-commerce applications.
Neural Personalized Topic Modeling for Mining User Preferences on Social Media
Authors: Luyang Liu, Qunyang Lin, Haonan Tong, Hongyin Zhu, Ke Liu, Min Wang, Chuang Zhang
Venue: CIKM
Year: 2023

With the rapid development of web services, social media has been a prevalent and readily way for people to express themselves and share their daily lives. Consequently, numerous user-generated content is accumulated on social media platforms. These data usually contain rich information and knowledge for users, which is a viable source for user data mining. As one of the prevalent techniques in user data mining, mining personalized topics and discovering user preferences from social media data attract much interest in academic and industrial communities. The emerging Neural Topic Models(NTMs) have recently shown leading performance and scalability by employing neural networks. However, most existing NTMs usually model topics simply from observed document token information and do not explicitly take user preferences into the generative process, which inevitably fails to model personalized topics. To address this issue, we introduce Neural Personalized Topic Model(NPTM), a novel NTM that can discover personalized topics and user preferences. NPTM introduces a novel hybrid generative process for combining user preferences and contextualized document codes in modeling personalized topics. A transformer-based document encoder to obtain contextualized document codes. For user preference modeling, NPTM regards user-related information as trainable user embeddings, further determining user preferences over the topics. Following the proposed hybrid generative process, we present a module-wise asynchronous optimization strategy to get coherent topics and user preferences. Then, we apply our model to two challenging real-world social media post collections and compare them against several baseline methods to verify our contributions. The experimental results demonstrate the effectiveness of the proposed method.
Boosting Meta-Learning Cold-Start Recommendation with Graph Neural Network
Authors: Han Liu, Hongxiang Lin, Xiaotong Zhang, Fenglong Ma, Hongyang Chen, Lei Wang, Hong Yu, Xianchao Zhang
Venue: CIKM
Year: 2023

Meta-learning methods have shown to be effective in dealing with cold-start recommendation. However, most previous methods rely on an ideal assumption that there exists a similar data distribution between source and target tasks, which are unsuitable for the scenario that only extremely limited number of new user or item interactions are available. In this paper, we propose to boost meta-learning cold-start recommendation with graph neural network (MeGNN). First, it utilizes the global neighborhood translation learning to obtain consistent potential interactions for all new user and item nodes, which can refine their representations. Second, it employs the local neighborhood translation learning to predict specific potential interactions for each node, thus guaranteeing the personalized requirement. In experiments, we combine MeGNN with two representative meta-learning models MeLU and TaNP. Extensive results on two widely-used datasets show the superiority of MeGNN in four different scenarios.
Understanding the Multi-vector Dense Retrieval Models
Authors: Qi Liu, Jiaxin Mao
Venue: CIKM
Year: 2023

While dense retrieval has become a promising alternative to the traditional text retrieval models, such as BM25, some recent studies show that multi-vector dense retrieval models are more effective than the single-vector method in retrieval tasks. However, due to a lack of interpretability, why the multi-vector method outperforms its single-vector counterpart has not been fully studied. To fill this research gap, in this work, we investigate and compare the behaviors of single-vector and multi-vector models in retrieval. Specifically, we analyze the vocabulary distribution of dense representations by mapping them back to the sparse, vocabulary space. Our empirical findings show that the multi-vector representation has more lexical overlaps between queries and passages. Additionally, we show that this feature of multi-vector representation can enhance its ranking performance when a given passage can fulfill different information needs and thus can be retrieved by different queries. These results shed light on the internal mechanisms of multi-vector representation and may provide new perspectives for future research.
SMEF: Social-aware Multi-dimensional Edge Features-based Graph Representation Learning for Recommendation
Authors: Xiao Liu, Shunmei Meng, Qianmu Li, Lianyong Qi, Xiaolong Xu, Wanchun Dou, Xuyun Zhang
Venue: CIKM
Year: 2023

Exploring user-item interaction cues is crucial for the performance of recommender systems. Explicit investigation of interaction cues is made possible by using graph-based models, where each user-item relationship is described by an edge, and the introduction of user-user social network. While existing graph-based recommendation methods use only a single-value edge to define the relationship between a pair of user and item, which limits the ability to represent complex user-item interactions. Furthermore, some social recommendation methods overlook the heterogeneous user behavior patterns in social and interaction relationships, resulting in the suboptimal performance of existing systems. In this paper, we propose a novel Social-aware Multi-dimensional Edge Feature-based Graph Representation Learning method, called SMEF. It represents all users and items as a graph and deep learns a multi-dimensional edge feature to explicitly describe the task-specific relationships of each user-item pair. Specifically, the proposed SMEF focuses on two distinct user behavior patterns toward social friends and interactive items, which explore the underlying heterogeneous relationship cues within them. This way, the learned multi-dimensional edge features encode user information from both social and interaction aspects. The proposed SMEF is a plug-and-play module that can be combined with different recommendation frameworks and Graph Neural Networks (GNNs) backbones to generate high quality user representations. The experimental results achieved on three publicly accessible datasets show that our SMEF-based method outperforms strong baselines.
TemDep: Temporal Dependency Priority for Multivariate Time Series Prediction
Authors: Shu Liu, Jiaheng Wang, Jiamin Chen, Jianliang Gao, Yuhui Zhong
Venue: CIKM
Year: 2023

The multivariate fusion transformation is ubiquitous in multivariate time series prediction (MTSP) problems. The previous multivariate fusion transformation fuses the feature of different variates at a time step, then projects them to a new feature space for effective feature representation. However, temporal dependency is the most fundamental property of time series. The previous manner fails to capture the temporal dependency of the feature, which is destroyed in the transformed feature matrix. Multivariate feature extraction based on the feature matrix with missing temporal dependency leads to the loss of predictive performance of MTSP. To address this problem, we propose the Temporal Dependency Priority for Multivariate Time Series Prediction (TemDep) method. Specifically, TemDep extracts feature temporal dependency of multivariate time series first and then considers multivariate feature fusion. Moreover, the low-dimensional and high-dimensional feature fusion manners are designed with the temporal dependency priority to fit different dimensional multivariate time series. The extensive experimental results of different datasets show that our proposed method can outperform all state-of-the-art baseline methods. It proves the significance of temporal dependency priority for MTSP.
Khronos: A Real-Time Indexing Framework for Time Series Databases on Large-Scale Performance Monitoring Systems
Authors: Xinyu Liu, Zijing Wei, Wenqing Yu, Shaozhi Liu, Gang Wang, Xiaoguang Liu, Yusen Li
Venue: CIKM
Year: 2023

Time series databases play a critical role in large-scale performance monitoring systems. Metrics are required to be observable immediately after being generated to support real-time analysis. However, the commonly used Log-Structured Merge-Tree structure suffers from periodically visible delay spikes when a new segment is created due to the instantaneous index construction pressure. In this paper, we present Khronos, an asynchronous indexing framework tailored for high-cardinal monitoring data, aiming at reducing the visible delay. Firstly, we analyze the temporal locality nature of time series and propose a complementary index construction algorithm by only indexing series not reported before to relieve indexing workload. Secondly, we design index structures based on Minimum Excluded value function to effectively reuse indexes of previous segments. Thirdly, we take advantage of the non-repetitive feature of complementary indexes and further develop an intermediate query results reusing approach for deduplicating index traversal among segments. Moreover, we propose an index dependency management strategy that cuts off the previous reusing dependency before persistence to avoid extended dependency overhead. Experimental results show that our framework significantly reduces the visible delay from minutes to milliseconds. Khronos outperforms the state-of-the-art databases InfluxDB and TimeScaleDB with at least 4 times higher write throughput, hundreds of times lower visible delay, and 6 times lower query latency. Khronos has been deployed in production since 2020 and has become the largest performance monitoring database in Alibaba.
Quantifying the Effectiveness of Advertising: A Bootstrap Proportion Test for Brand Lift Testing
Authors: Wanjun Liu, Xiufan Yu, Jialiang Mao, Xiaoxu Wu, Justin Dyer
Venue: CIKM
Year: 2023

Brand Lift test is a widely deployed statistical tool for measuring the effectiveness of online advertisements on brand perception such as ad recall, brand familiarity and favorability. By formulating the problem of interest into a two-sample test on the binomial proportions from the control group (p_0) and the treatment group (p_1), Brand Lift test evaluates ads impact based on the statistical significance of test results. Traditional approaches construct the test statistics based on the absolute difference between the two observed proportions, a.k.a, absolute lift. In this work, we propose a new bootstrap test based on the percentage difference between the two observed proportions, i.e., relative lift. We provide rigorous theoretical guarantees on the asymptotic validity of the proposed relative-lift-based test. Our numerical studies suggest that the relative-lift-based test requires less stringent conditions than the absolute-lift-based test for controlling the type-I error rate. Interestingly, we also prove that the relative-lift-based test is more powerful than the absolute-lift-based test when the alternative is positive (i.e., p1 - p0 > 0), but less powerful when the alternative is negative (i.e., p1 - p0 < 0). The empirical performance of the proposed test is demonstrated by extensive simulation studies, an application to a publicly available A/B testing dataset from advertising, and real datasets collected from the Brand Lift Testing platform at LinkedIn.
Retrieval-Based Unsupervised Noisy Label Detection on Text Data
Authors: Peiyang Liu, Jinyu Yang, Lin Wang, Sen Wang, Yunlai Hao, Huihui Bai
Venue: CIKM
Year: 2023

The success of deep neural networks hinges on both high-quality annotations and copious amounts of data; however, in practice, a compromise between dataset size and quality frequently arises. Data collection and cleansing are often resource-intensive and time-consuming, leading to real-world datasets containing label noise that can introduce incorrect correlation patterns, adversely affecting model generalization capabilities. The efficient identification of corrupted patterns is indispensable, with prevalent methods predominantly concentrating on devising robust training techniques to preclude models from internalizing these patterns. Nevertheless, these supervised approaches often necessitate tailored training procedures, potentially resulting in overfitting corrupted patterns and a decline in detection performance. This paper presents a retrieval-based unsupervised solution for the detection of noisy labels, surpassing the performance of three current competitive methods in this domain.
Datasets and Interfaces for Benchmarking Heterogeneous Graph Neural Networks
Authors: Yijian Liu, Hongyi Zhang, Cheng Yang, Ao Li, Yugang Ji, Luhao Zhang, Tao Li, Jinyu Yang, Tianyu Zhao, Juan Yang, Hai Huang, Chuan Shi
Venue: CIKM
Year: 2023

In recent years, Heterogeneous Graph Neural Networks (HGNNs) have gained increasing attention due to their excellent performance in applications. However, the lack of high-quality benchmarks in new fields has become a critical limitation for developing and applying HGNNs. To accommodate the urgent need for emerging fields and the advancement of HGNNs, we present two large-scale, real-world, and challenging heterogeneous graph datasets from real scenarios: risk commodity detection and takeout recommendation. Meanwhile, we establish standard benchmark interfaces that provide over 40 heterogeneous graph datasets. We provide initial data split, unified evaluation metrics, and baseline results for future work, making it fair and handy to explore state-of-the-art HGNNs. Our interfaces also offer a comprehensive toolkit to research the characteristics of graph datasets. The above new datasets are publicly available on https://zenodo.org/communities/hgd, and the interface codes are available at https://github.com/BUPT-GAMMA/hgbi.
Counterfactual Adversarial Learning for Recommendation
Authors: Jialin Liu, Zijian Zhang, Xiangyu Zhao, Jun Li
Venue: CIKM
Year: 2023

Long-term user responses, i.e., clicks or purchases on e-commerce platforms, are crucial for sequential recommender systems. Recent off-policy evaluation methods involve these responses by simultaneously maximizing expected cumulative rewards. However, two aspects of these methods require further consideration. Firstly, from the system's point of view, candidates with various values are interchangeable, which may result in contradictory future recommendations despite having the same interaction history. Secondly, rewards are manually designed, which necessitates a trial-and-error approach to strike a balance between training stabilization and reward distinction. To address these issues, we propose a new sequential recommender system called NCM4Rec. Specifically, for the distinction problem, NCM4Rec achieves counterfactual consistency via a neural causal model, which is learnable yet equally expressive as classic structural causal models. Such consistency is maintained by a Gumbel-Max design. For the representing problem, NCM4Rec encodes different types of responses as one-hot vectors and captures the long-term preference via adversarial learning. As a consequence, NCM4Rec is both adaptive and identifiable. Both theoretical analyses of the consistency and empirical studies over two real-world datasets demonstrate the effectiveness of our method.