OpenRaiser/Intern-Atlas huggingface dataset:
LOAM: Improving Long-tail Session-based Recommendation via Niche Walk Augmentation and Tail Session Mixup
Authors: Heeyoon Yang, YunSeok Choi, Gahyung Kim, Jee-Hyong Lee
Venue: Conference SIGIR 2023
Year: 2023

Session-based recommendation aims to predict the user's next action based on anonymous sessions without using side information. Most of the real-world session datasets are sparse and have long-tail item distribution. Although long-tail item recommendation plays a crucial role in improving user satisfaction, only a few methods have been proposed to take the long-tail session recommendation into consideration. Previous works in handling data sparsity problems are mostly limited to self-supervised learning techniques with heuristic augmentation which can ruin the original characteristic of session datasets, sequential and co-occurrences, and make noisier short sessions by dropping items and cropping sequences. We propose a novel method, LOAM, improving LOng-tail session-based recommendation via niche walk Augmentation and tail session Mixup, that alleviates popularity bias and enhances long-tail recommendation performance. LOAM consists of two modules, Niche Walk Augmentation (NWA) and Tail Session Mixup (TSM). NWA can generate synthetic sessions considering long-tail distribution which are likely to be found in original datasets, unlike previous heuristic methods, and expose a recommender model to various item transitions with global information. This improves the item coverage of recommendations. TSM makes the model more generalized and robust by interpolating sessions at the representation level. It encourages the recommender system to predict niche items with more diversity and relevance. We conduct extensive experiments with four real-world datasets and verify that our methods greatly improve tail performance while balancing overall performance.
Ensemble pruning via individual contribution ordering
Authors: Zhenyu Lu, Xindong Wu, Xingquan Zhu, Josh C. Bongard
Venue: Conference KDD 2010
Year: 2010

An ensemble is a set of learned models that make decisions collectively. Although an ensemble is usually more accurate than a single learner, existing ensemble methods often tend to construct unnecessarily large ensembles, which increases the memory consumption and computational cost. Ensemble pruning tackles this problem by selecting a subset of ensemble members to form subensembles that are subject to less resource consumption and response time with accuracy that is similar to or better than the original ensemble. In this paper, we analyze the accuracy/diversity trade-off and prove that classifiers that are more accurate and make more predictions in the minority group are more important for subensemble construction. Based on the gained insights, a heuristic metric that considers both accuracy and diversity is proposed to explicitly evaluate each individual classifier's contribution to the whole ensemble. By incorporating ensemble members in decreasing order of their contributions, subensembles are formed such that users can select the top $p$ percent of ensemble members, depending on their resource availability and tolerable waiting time, for predictions. Experimental results on 26 UCI data sets show that subensembles formed by the proposed EPIC (Ensemble Pruning via Individual Contribution ordering) algorithm outperform the original ensemble and a state-of-the-art ensemble pruning method, Orientation Ordering (OO).
Addressing heterophily in graph anomaly detection: A perspective of graph spectrum
Authors: Yuan Gao, Xiang Wang, Xiangnan He, Zhenguang Liu, Huamin Feng, Yongdong Zhang
Venue: Conference WWW 2023
Year: 2023

Graph anomaly detection (GAD) suffers from heterophily — abnormal nodes are sparse so that they are connected to vast normal nodes. The current solutions upon Graph Neural Networks (GNNs) blindly smooth the representation of neiboring nodes, thus undermining the discriminative information of the anomalies. To alleviate the issue, recent studies identify and discard inter-class edges through estimating and comparing the node-level representation similarity. However, the representation of a single node can be misleading when the prediction error is high, thus hindering the performance of the edge indicator. In graph signal processing, the smoothness index is a widely adopted metric which plays the role of frequency in classical spectral analysis. Considering the ground truth Y to be a signal on graph, the smoothness index is equivalent to the value of the heterophily ratio. From this perspective, we aim to address the heterophily problem in the spectral domain. First, we point out that heterophily is positively associated with the frequency of a graph. Towards this end, we could prune inter-class edges by simply emphasizing and delineating the high-frequency components of the graph. Recall that graph Laplacian is a high-pass filter, we adopt it to measure the extent of 1-hop label changing of the center node and indicate high-frequency components. As GAD can be formulated as a semi-supervised binary classification problem, only part of the nodes are labeled. As an alternative, we use the prediction of the nodes to estimate it. Through our analysis, we show that prediction errors are less likely to affect the identification process. Extensive empirical evaluations on four benchmarks demonstrate the effectiveness of the indicator over popular homophilic, heterophilic, and tailored fraud detection methods. Our proposed indicator can effectively reduce the heterophily degree of the graph, thus boosting the overall GAD performance. Codes are open-sourced in https://github.com/blacksingular/GHRN.
THGNN: an embedding-based model for anomaly detection in dynamic heterogeneous social networks
Authors: Yilin Li, Jiaqi Zhu, Congcong Zhang, Yi Yang, Jiawen Zhang, Ying Qiao, Hongan Wang
Venue: Conference CIKM 2023
Year: 2023

Anomaly detection, particularly the detection of anomalous behaviors in dynamic and heterogeneous social networks, is becoming more and more crucial in real life. Traditional rule-based and feature-based methods cannot well capture the structural and temporal patterns of ever-changing user behaviors. Moreover, most of the existing works based on network embedding either rely on discretized snapshots, which have ignored accurate temporal relations among user behaviors and weakened the impact of new edges, or fail to utilize dynamic and heterogeneous information simultaneously to distinguish varying effects of new edges on existing nodes. In this paper, we propose an end-to-end continuous-time model, named Temporal Heterogeneous Graph Neural Network (THGNN), to detect anomalous behaviors (edges) in dynamic heterogeneous social networks. Specifically, the model constantly updates node embeddings by propagating the information of a new edge to its source and target nodes as well as their neighbors. In this process, heterogeneous encoders are employed to handle different types of nodes and edges. What is more, a novel dual-level distributive attention mechanism is designed to allocate the influence degree of a currently interacting node to its multiple neighbors, considering the combined effect of edge type and time interval information. That can be regarded as an extension of the classical aggregative attention mechanism in the opposite direction. Extensive experiments on four real-world datasets demonstrate that THGNN outperforms all the baselines on the task of anomalous edge detection, achieving an average AUC gain of 6% across all datasets.
Sequence As Genes: An User Behavior Modeling Framework for Fraud Transaction Detection in E-commerce
Authors: Ziming Wang, Qianru Wu, Baolin Zheng, Junjie Wang, Kaiyu Huang, Yanjie Shi
Venue: Conference KDD 2023
Year: 2023

With the explosive growth of e-commerce, detecting fraudulent transactions in real-world scenarios is becoming increasingly important for e-commerce platforms. Recently, several supervised approaches have been proposed to use user behavior sequences, which record the user's track on platforms and contain rich information for fraud transaction detection. Nevertheless, these methods always suffer from the scarcity of labeled data in real-world scenarios. The recent remarkable pre-training methods in Natural Language Processing (NLP) and Computer Vision (CV) domains offered glimmers of light. However, user behavior sequences differ intrinsically from text, images, and videos. In this paper, we propose a novel and general user behavior pre-training framework, named Sequence As GEnes (SAGE), which provides a new perspective for user behavior modeling. Following the inspiration of treating sequences as genes, we carefully designed the user behavior data organization paradigm and pre-training scheme. Specifically, we propose an efficient data organization paradigm inspired by the nature of DNA expression, which decouples the length of behavior sequences and the corresponding time spans. Also inspired by the natural mechanisms in genetics, we propose two pre-training tasks, namely sequential mutation and sequential recombination, to improve the robustness and consistency of user behavior representations in complicated real-world scenes. Extensive experiments on four differentiated fraud transaction detection real scenarios demonstrate the effectiveness of our proposed framework.
MIDLG: Mutual Information based Dual Level GNN for Transaction Fraud Complaint Verification
Authors: Wen Zheng, Bingbing Xu 0001, Emiao Lu, Yang Li 0202, Qi Cao 0005, Xuan Zong 0001, Huawei Shen
Venue: Conference KDD 2023
Year: 2023

"Transaction fraud" complaint verification, i.e., verifying whether a transaction corresponding to a complaint is fraudulent, is particularly critical to prevent economic loss. Compared with traditional fraud pre-transaction detection, complaint verification puts forward higher requirements: 1)an individual tends to exhibit different identities in different complaints, e.g., complainant or respondent, requiring the model to capture identity-related representations corresponding to the complaint; 2)the fraud ways evolve frequently to confront detection, requiring the model to perform stably under different fraud ways. Previous methods mainly focused on fraud pre-transaction detection, utilizing the historical information of users or conduct message passing based GNNs on relationship networks. However, they rarely consider capturing various identity-related representations and ignore the evolution of fraud ways, leading to failure in complaint verification. To address the above challenges, we propose the mutual information based dual level graph neural network, namely MIDLG, which defines a complaint as a super-node consisting of involved individuals, and characterizes the individual over node-level and super-node-level. Furthermore, the mutual information minimization objective is proposed based on "complaint verification-causal graph" to decouple the model prediction from relying on specific fraud ways, and thus achieve stability. MIDLG achieves SOTA results through extensive experiments in complaint verification on WeChat Finance, one online payment service serving more than 600 million users in China.
Mining Dual Emotion for Fake News Detection
Authors: Xueyao Zhang, Juan Cao, Xirong Li, Qiang Sheng, Lei Zhong, Kai Shu
Venue: Conference WWW 2021
Year: 2021

Emotion plays an important role in detecting fake news online. When leveraging emotional signals, the existing methods focus on exploiting the emotions of news contents that conveyed by the publishers (i.e., publisher emotion). However, fake news often evokes high-arousal or activating emotions of people, so the emotions of news comments aroused in the crowd (i.e., social emotion) should not be ignored. Furthermore, it remains to be explored whether there exists a relationship between publisher emotion and social emotion (i.e., dual emotion), and how the dual emotion appears in fake news. In this paper, we verify that dual emotion is distinctive between fake and real news and propose Dual Emotion Features to represent dual emotion and the relationship between them for fake news detection. Further, we exhibit that our proposed features can be easily plugged into existing fake news detectors as an enhancement. Extensive experiments on three real-world datasets (one in English and the others in Chinese) show that our proposed feature set: 1) outperforms the state-of-the-art task-related emotional features; 2) can be well compatible with existing fake news detectors and effectively improve the performance of detecting fake news.1 2
RicciNet: Deep clustering via a Riemannian generative model
Authors: Li Sun, Jingbin Hu, Suyang Zhou, Zhenhao Huang, Junda Ye, Hao Peng, Zhengtao Yu, Philip S. Yu
Venue: Conference WWW 2024
Year: 2024

In recent years, deep clustering has achieved encouraging results. However, existing deep clustering methods work with the traditional Euclidean space and thus present deficiency on clustering complex structures. On the contrary, Riemannian geometry provides an elegant framework to model complex structures as well as a powerful tool for clustering, i.e., the Ricci flow. In this paper, we rethink the problem of deep clustering, and introduce the Riemannian geometry to deep clustering for the first time. Deep clustering in Riemannian manifold still faces significant challenges: (1) Ricci flow itself is unaware of cluster membership, (2) Ricci curvature prevents the gradient backpropagation, and (3) learning the flow largely remains open in the manifold. To bridge these gaps, we propose a novel Riemannian generative model (RicciNet), a neural Ricci flow with several theoretical guarantees. The novelty is that we model the dynamic self-clustering process of Ricci flow: data points move to the respective clusters in the manifold, influenced by Ricci curvatures. The point's trajectory is characterized by a parametric velocity, taking the form of Ordinary Differential Equation (ODE). Specifically, we encode data points as samples of Gaussian mixture in the manifold where we propose two types of reparameterization approaches: Gumbel reparameterization, and geometric trick. We formulate a differentiable Ricci curvature parameterized by a Riemannian graph convolution. Thereafter, we propose a geometric learning approach in which we study the geometric regularity of the point's trajectory, and learn the flow via distance matching and velocity matching. Consequently, data points go along the shortest Ricci flow to complete clustering. Extensive empirical results show RicciNet outperforms Euclidean deep methods.
DDPO: Direct Dual Propensity Optimization for Post-Click Conversion Rate Estimation
Authors: Hongzu Su, Lichao Meng, Lei Zhu, Ke Lu, Jingjing Li
Venue: Conference SIGIR 2024
Year: 2024

In online advertising, the sample selection bias problem is a major cause of inaccurate conversion rate estimates. Current mainstream solutions only perform causality-based optimization in the click space since the conversion labels in the non-click space are absent. However, optimization for unclicked samples is equally essential because the non-click space contains more samples and user characteristics than the click space. To exploit the unclicked samples, we propose a Direct Dual Propensity Optimization (DDPO) framework to optimize the model directly in impression space with both clicked and unclicked samples. In this framework, we specifically design a click propensity network and a conversion propensity network. The click propensity network is dedicated to ensuring that optimization in the click space is unbiased. The conversion propensity network is designed to generate pseudo-conversion labels for unclicked samples, thus overcoming the challenge of absent labels in non-click space. With these two propensity networks, we are able to perform causality-based optimization in both click space and non-click space. In addition, to strengthen the causal relationship, we design two causal transfer modules for the conversion rate prediction model with the attention mechanism. The proposed framework is evaluated on five real-world public datasets and one private Tencent advertising dataset. Experimental results verify that our method is able to improve the prediction performance significantly. For instance, our method outperforms the previous state-of-the-art method by 7.0% in terms of the Area Under the Curve on the Ali-CCP dataset.
ESCM2: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation
Authors: Hao Wang, Tai-Wei Chang, Tianqiao Liu, Jianmin Huang, Zhichao Chen, Chao Yu, Ruopeng Li, Wei Chu
Venue: Conference SIGIR 2022
Year: 2022

Accurate estimation of post-click conversion rate is critical for building recommender systems, which has long been confronted with sample selection bias and data sparsity issues. Methods in the Entire Space Multi-task Model (ESMM) family leverage the sequential pattern of user actions, \ie $impression\rightarrow click \rightarrow conversion$ to address data sparsity issue. However, they still fail to ensure the unbiasedness of CVR estimates. In this paper, we theoretically demonstrate that ESMM suffers from the following two problems: (1) Inherent Estimation Bias (IEB) for CVR estimation, where the CVR estimate is inherently higher than the ground truth; (2) Potential Independence Priority (PIP) for CTCVR estimation, where ESMM might overlook the causality from click to conversion. To this end, we devise a principled approach named Entire Space Counterfactual Multi-task Modelling (ESCM$^2$), which employs a counterfactual risk miminizer as a regularizer in ESMM to address both IEB and PIP issues simultaneously. Extensive experiments on offline datasets and online environments demonstrate that our proposed ESCM$^2$ can largely mitigate the inherent IEB and PIP issues and achieve better performance than baseline models.
Unbiased sequential recommendation with latent confounders
Authors: Zhenlei Wang, Shiqi Shen, Zhipeng Wang, Bo Chen, Xu Chen, Ji-Rong Wen
Venue: Conference WWW 2022
Year: 2022

Sequential recommendation holds the promise of understanding user preference by capturing successive behavior correlations. Existing research focus on designing different models for better fitting the offline datasets. However, the observational data may have been contaminated by the exposure or selection biases, which renders the learned sequential models unreliable. In order to solve this fundamental problem, in this paper, we propose to reformulate the sequential recommendation task with the potential outcome framework, where we are able to clearly understand the data bias mechanism and correct it by re-weighting the training instances with the inverse propensity score (IPS). For more robustness modeling, a clipping strategy is applied to the IPS estimation to reduce the variance of the learning objective. To make our framework more practical, we design a parameterized model to remove the impact of the potential latent confounders. At last, we theoretically analyze the unbiasedness of the proposed framework under both vanilla and clipping IPS estimations. To the best of our knowledge, this is the first work on debiased sequential recommendation. We conduct extensive experiment based on both synthetic and real-world datasets to demonstrate the effectiveness of our framework.
Ukd: Debiasing conversion rate estimation via uncertainty-regularized knowledge distillation
Authors: Zixuan Xu, Penghui Wei, Weimin Zhang, Shaoguo Liu, Liang Wang, Bo Zheng
Venue: Conference WWW 2022
Year: 2022

In online advertising, conventional post-click conversion rate (CVR) estimation models are trained using clicked samples. However, during online serving the models need to estimate for all impression ads, leading to the sample selection bias (SSB) issue. Intuitively, providing reliable supervision signals for unclicked ads is a feasible way to alleviate the SSB issue. This paper proposes an uncertainty-regularized knowledge distillation (UKD) framework to debias CVR estimation via distilling knowledge from unclicked ads. A teacher model learns click-adaptive representations and produces pseudo-conversion labels on unclicked ads as supervision signals. Then a student model is trained on both clicked and unclicked ads with knowledge distillation, performing uncertainty modeling to alleviate the inherent noise in pseudo-labels. Experiments on billion-scale datasets show that UKD outperforms previous debiasing methods. Online results verify that UKD achieves significant improvements.
Fedmultimodal: A benchmark for multimodal federated learning
Authors: Tiantian Feng, Digbalay Bose, Tuo Zhang, Rajat Hebbar, Anil Ramakrishna, Rahul Gupta 0001, Mi Zhang 0002, Salman Avestimehr, Shrikanth Narayanan
Venue: Conference KDD 2023
Year: 2023

Over the past few years, Federated Learning (FL) has become an emerging machine learning technique to tackle data privacy challenges through collaborative training. In the Federated Learning algorithm, the clients submit a locally trained model, and the server aggregates these parameters until convergence. Despite significant efforts that have been made to FL in fields like computer vision, audio, and natural language processing, the FL applications utilizing multimodal data streams remain largely unexplored. It is known that multimodal learning has broad real-world applications in emotion recognition, healthcare, multimedia, and social media, while user privacy persists as a critical concern. Specifically, there are no existing FL benchmarks targeting multimodal applications or related tasks. In order to facilitate the research in multimodal FL, we introduce FedMultimodal, the first FL benchmark for multimodal learning covering five representative multimodal applications from ten commonly used datasets with a total of eight unique modalities. FedMultimodal offers a systematic FL pipeline, enabling end-to-end modeling framework ranging from data partition and feature extraction to FL benchmark algorithms and model evaluation. Unlike existing FL benchmarks, FedMultimodal provides a standardized approach to assess the robustness of FL against three common data corruptions in real-life multimodal applications: missing modalities, missing labels, and erroneous labels. We hope that FedMultimodal can accelerate numerous future research directions, including designing multimodal FL algorithms toward extreme data heterogeneity, robustness multimodal FL, and efficient multimodal FL. The datasets and benchmark results can be accessed at: https://github.com/usc-sail/fed-multimodal.
Efficient and information-preserving future frame prediction and beyond
Authors: Wei Yu 0001, Yichao Lu, Steve Easterbrook, Sanja Fidler
Venue: Conference ICLR 2020
Year: 2020

Applying resolution-preserving blocks is a common practice to maximize information preservation in video prediction, yet their high memory consumption greatly limits their application scenarios. We propose CrevNet, a Conditionally Reversible Network that uses reversible architectures to build a bijective two-way autoencoder and its complementary recurrent predictor. Our model enjoys the theoretically guaranteed property of no information loss during the feature extraction, much lower memory consumption and computational efficiency. The lightweight nature of our model enables us to incorporate 3D convolutions without concern of memory bottleneck, enhancing the model's ability to capture both short-term and long-term temporal dependencies. Our proposed approach achieves state-of-the-art results on Moving MNIST, Traffic4cast and KITTI datasets. We further demonstrate the transferability of our self-supervised learning method by exploiting its learnt features for object detection on KITTI. Our competitive results indicate the potential of using CrevNet as a generative pre-training strategy to guide downstream tasks.