OpenRaiser/Intern-Atlas huggingface dataset:
Hypergraph random walks, laplacians, and clustering
Authors: Koby Hayashi, Sinan G. Aksoy, Cheong Hee Park, Haesun Park
Venue: Conference CIKM 2020
Year: 2020

We propose a flexible framework for clustering hypergraph-structured data based on recently proposed random walks utilizing edge-dependent vertex weights. When incorporating edge-dependent vertex weights (EDVW), a weight is associated with each vertex-hyperedge pair, yielding a weighted incidence matrix of the hypergraph. Such weightings have been utilized in term-document representations of text data sets. We explain how random walks with EDVW serve to construct different hypergraph Laplacian matrices, and then develop a suite of clustering methods that use these incidence matrices and Laplacians for hypergraph clustering. Using several data sets from real-life applications, we compare the performance of these clustering algorithms experimentally against a variety of existing hypergraph clustering methods. We show that the proposed methods produce higher-quality clusters and conclude by highlighting avenues for future work.
Unified pretraining for recommendation via task hypergraphs
Authors: Mingdai Yang, Zhiwei Liu, Liangwei Yang, Xiaolong Liu, Chen Wang, Hao Peng, Philip S. Yu
Venue: Conference WSDM 2024
Year: 2024

Although pretraining has garnered significant attention and popularity in recent years, its application in graph-based recommender systems is relatively limited. It is challenging to exploit prior knowledge by pretraining in widely used ID-dependent datasets. On the one hand, user-item interaction history in one dataset can hardly be transferred to other datasets through pretraining, where IDs are different. On the other hand, pretraining and finetuning on the same dataset leads to a high risk of overfitting. In this paper, we propose a novel multitask pretraining framework named Unified Pretraining for Recommendation via Task Hypergraphs. For a unified learning pattern to handle diverse requirements and nuances of various pretext tasks, we design task hypergraphs to generalize pretext tasks to hyperedge prediction. A novel transitional attention layer is devised to discriminatively learn the relevance between each pretext task and recommendation. Experimental results on three benchmark datasets verify the superiority of UPRTH. Additional detailed investigations are conducted to demonstrate the effectiveness of the proposed framework.
Gbert: pre-training user representations for ephemeral group recommendation
Authors: Song Zhang, Nan Zheng, Danli Wang
Venue: Conference CIKM 2022
Year: 2022

Due to the prevalence of group activities on social networks, group recommendations have received an increasing number of attentions. Most group recommendation methods concentrated on dealing with persistent groups, while little attention has paid to ephemeral groups. Ephemeral groups are formed ad-hoc for one-time activities, and therefore they suffer severely from data sparsity and cold-start problems. To deal with such problems, we propose a pre-training and fine-tuning method called GBERT for improved group recommendations, which employs BERT to enhance the expressivity and capture group-specific preferences of members. In the pre-training stage, GBERT employs three pre-training tasks to alleviate data sparsity and cold-start problem, and learn better user representations. In the fine-tuning stage, an influence-based regulation objective is designed to regulate user and group representations by allocating weights according to each member's influence. Extensive experiments on three public datasets demonstrate its superiority over the state-of-the-art methods for ephemeral group recommendations.
Adpl: Adversarial prompt-based domain adaptation for dialogue summarization with knowledge disentanglement
Authors: Lulu Zhao, Fujia Zheng, Weihao Zeng, Keqing He, Ruotong Geng, Huixing Jiang, Wei Wu, Weiran Xu
Venue: Conference SIGIR 2022
Year: 2022

Traditional dialogue summarization models rely on a large-scale manually-labeled corpus, lacking generalization ability to new domains, and domain adaptation from a labeled source domain to an unlabeled target domain is important in practical summarization scenarios. However, existing domain adaptation works in dialogue summarization generally require large-scale pre-training using extensive external data. To explore the lightweight fine-tuning methods, in this paper, we propose an efficient Adversarial Disentangled Prompt Learning (ADPL) model for domain adaptation in dialogue summarization. We introduce three kinds of prompts including domain-invariant prompt (DIP), domain-specific prompt (DSP), and task-oriented prompt (TOP). DIP aims to disentangle and transfer the shared knowledge from the source domain and target domain in an adversarial way, which improves the accuracy of prediction about domain-invariant information and enhances the ability for generalization to new domains. DSP is designed to guide our model to focus on domain-specific knowledge using domain-related features. TOP is to capture task-oriented knowledge to generate high-quality summaries. Instead of fine-tuning the whole pre-trained language model (PLM), we only update the prompt networks but keep PLM fixed. Experimental results on the zero-shot setting show that the novel design of prompts can yield more coherent, faithful, and relevant summaries than baselines using the prefix-tuning, and perform at par with fine-tuning while being more efficient. Overall, our work introduces a prompt-based perspective to the zero-shot learning for dialogue summarization task and provides valuable findings and insights for future research.
CDRNP: Cross-Domain Recommendation to Cold-Start Users via Neural Process
Authors: Xiaodong Li, Jiawei Sheng, Jiangxia Cao, Wenyuan Zhang, Quangang Li, Tingwen Liu
Venue: Conference WSDM 2024
Year: 2024

Cross-domain recommendation (CDR) has been proven as a promising way to tackle the user cold-start problem, which aims to make recommendations for users in the target domain by transferring the user preference derived from the source domain. Traditional CDR studies follow the embedding and mapping (EMCDR) paradigm, which transfers user representations from the source to target domain by learning a user-shared mapping function, neglecting the user-specific preference. Recent CDR studies attempt to learn user-specific mapping functions in meta-learning paradigm, which regards each user's CDR as an individual task, but neglects the preference correlations among users, limiting the beneficial information for user representations. Moreover, both of the paradigms neglect the explicit user-item interactions from both domains during the mapping process. To address the above issues, this paper proposes a novel CDR framework with neural process (NP), termed as CDRNP. Particularly, it develops the meta-learning paradigm to leverage user-specific preference, and further introduces a stochastic process by NP to capture the preference correlations among the overlapping and cold-start users, thus generating more powerful mapping functions by mapping the user-specific preference and common preference correlations to a predictive probability distribution. In addition, we also introduce a preference remainer to enhance the common preference from the overlapping users, and finally devises an adaptive conditional decoder with preference modulation to make prediction for cold-start users with items in the target domain. Experimental results demonstrate that CDRNP outperforms previous SOTA methods in three real-world CDR scenarios.
Cyclener: an unsupervised training approach for named entity recognition
Authors: Andrea Iovine, Anjie Fang, Besnik Fetahu, Oleg Rokhlenko, Shervin Malmasi
Venue: Conference WWW 2022
Year: 2022

Named Entity Recognition (NER) is a crucial natural language understanding task for many down-stream tasks such as question answering and retrieval. Despite significant progress in developing NER models for multiple languages and domains, scaling to emerging and/or low-resource domains still remains challenging, due to the costly nature of acquiring training data. We propose CycleNER, an unsupervised approach based on cycle-consistency training that uses two functions: (i) sentence-to-entity – S2E and (ii) entity-to-sentence – E2S, to carry out the NER task. CycleNER does not require annotations but a set of sentences with no entity labels and another independent set of entity examples. Through cycle-consistency training, the output from one function is used as input for the other (e.g. S2E → E2S) to align the representation spaces of both functions and therefore enable unsupervised training. Evaluation on several domains comparing CycleNER against supervised and unsupervised competitors shows that CycleNER achieves highly competitive performance with only a few thousand input sentences. We demonstrate competitive performance against supervised models, achieving 73% of supervised performance without any annotations on CoNLL03, while significantly outperforming unsupervised approaches.
Rumor detection on social media with event augmentations
Authors: Zhenyu He 0008, Ce Li 0003, Fan Zhou 0002, Yi Yang 0042
Venue: Conference SIGIR 2021
Year: 2021

With the rapid growth of digital data on the Internet, rumor detection on social media has been vital. Existing deep learning-based methods have achieved promising results due to their ability to learn high-level representations of rumors. Despite the success, we argue that these approaches require large reliable labeled data to train, which is time-consuming and data-inefficient. To address this challenge, we present a new solution, Rumor Detection on social media with Event Augmentations (RDEA), which innovatively integrates three augmentation strategies by modifying both reply attributes and event structure to extract meaningful rumor propagation patterns and to learn intrinsic representations of user engagement. Moreover, we introduce contrastive self-supervised learning for the efficient implementation of event augmentations and alleviate limited data issues. Extensive experiments conducted on two public datasets demonstrate that RDEA achieves state-of-the-art performance over existing baselines. Besides, we empirically show the robustness of RDEA when labeled data are limited.
Towards trustworthy rumor detection with interpretable graph structural learning
Authors: Leyuan Liu, Junyi Chen, Zhangtao Cheng, Wenxin Tai, Fan Zhou
Venue: Conference CIKM 2023
Year: 2023

The exponential growth of digital information has amplified the necessity for effective rumor detection on social media. However, existing approaches often neglect the inherent noise and uncertainty in rumor propagation, leading to obscure learning mechanisms. Moreover, current deep-learning methodologies, despite their top-tier performance, are heavily dependent on supervised learning, which is labor-intensive and inefficient. Their prediction credibility is also questionable. To tackle these issues, we present a new framework, TrustRD, for reliable rumor detection. Our framework incorporates a self-supervised learning module, designed to derive interpretable and informative representations with less reliance on large labeled data sets. A downstream model based on Bayesian networks, which is further refined with adversarial training, enhances performance while providing a quantifiable trustworthiness assessment of results. Our methods' effectiveness is confirmed through experiments on two benchmark datasets.
Generative-free urban flow imputation
Authors: Senzhang Wang, Jiyue Li, Hao Miao, Junbo Zhang, Junxing Zhu, Jianxin Wang
Venue: Conference CIKM 2022
Year: 2022

Urban flow imputation, which aims to infer the missing flows of some locations based on the available flows of surrounding areas, is critically important to various smart city related applications such as urban planning and public safety. Although many methods are proposed to impute time series data, they may not be feasible to be directly applied on urban flow data due to the following reasons. First, urban flows have the complex spatial and temporal correlations which are much harder to be captured compared with time series data. Second, the urban flow data can be random missing (i.e., missing randomly in terms of times and locations) or block missing (i.e., missing for all locations in a particular time slot). Thus it is difficult for existing methods to work well on both scenarios. In this paper, we for the first time study the urban flow imputation problem and propose a generative-free Attention-based Spatial-Temporal Combine and Mix Completion Network model (AST-CMCN for short) to effectively address it. Specifically, AST-CMCN consists of a Spatial and Temporal Completion Network (SATCNet for short) and a Spatial-Temporal Mix Completion Network (STMCNet for short). SATCNet is composed of stacked GRUAtt modules to capture the geographical and temporal correlations of the urban flows, separately. STMCNet is designed to capture the complex spatial-temporal associations jointly between historical urban flows and current data. A Message Passing module is also proposed to capture new spatial-temporal patterns that never appear in the historical data. Extensive experiments on two large real-world datasets validate the effectiveness and efficiency of our method compared with the state-of-the-art baselines.
Enhancing criminal case matching through diverse legal factors
Authors: Jie Zhao, Ziyu Guan, Wei Zhao, Yue Jiang
Venue: Conference SIGIR 2024
Year: 2024

Criminal case matching endeavors to determine the relevance between different criminal cases. Conventional methods predict the relevance solely based on instance-level semantic features and neglect the diverse legal factors (LFs), which are associated with diverse court judgments. Consequently, comprehensively representing a criminal case remains a challenge for these approaches. Moreover, extracting and utilizing these LFs for criminal case matching face two challenges: (1) the manual annotations of LFs rely heavily on specialized legal knowledge; (2) overlaps among LFs may potentially harm the model's performance. In this paper, we propose a two-stage framework named Diverse Legal Factor-enhanced Criminal Case Matching (DLF-CCM). Firstly, DLF-CCM employs a multi-task learning framework to pre-train an LF extraction network on a large-scale legal judgment prediction dataset. In stage two, DLF-CCM introduces an LF de-redundancy module to learn shared LF and exclusive LFs. Moreover, an entropy-weighted fusion strategy is introduced to dynamically fuse the multiple relevance generated by all LFs. Experimental results validate the effectiveness of DLF-CCM and show its significant improvements over competitive baselines. Code: https://github.com/jiezhao6/DLF-CCM.
User feedback and ranking in-a-loop: Towards self-adaptive dialogue systems
Authors: Chen Shi, Yuxiang Hu, Zengming Zhang, Liang Shao, Feijun Jiang
Venue: Conference SIGIR 2021
Year: 2021

Accurate skill retrieval is a key factor for the success of modern conversational AI agents. The major challenges lie in the ambiguity in human spoken language and the wide spectrum of candidate skills. In this paper, we make the first attempt to attack the problem by implementing a user feedback enhanced reranking strategy, and propose a self-adaptive dialogue system (AdaDial) for conversational AI agents. In AdaDial, we consider estimating user feedback and adjusting ranking strategy into a "closed-loop". In particular, we propose a scalable schema for user feedback estimation and a feedback enhanced reranking model with customized feature encoding, target attention based feature assembling, and multi-task learning. As a result, AdaDial achieves self-adaptivity at both individual- and system-levels. Online experimental results demonstrate that AdaDial could not only retrieve desired skills for different users in different scenarios, but also correct its regular strategy according to negative feedback. AdaDial has been deployed on a large-scale conversational AI agents with tens of millions daily queries, and is bringing continued positive impacts on user experience.
Few-shot insider threat detection
Authors: Shuhan Yuan, Panpan Zheng, Xintao Wu, Hanghang Tong
Venue: Conference CIKM 2020
Year: 2020

Insiders cause significant cyber-security threats to organizations. Due to a very limited number of insiders, most of the current studies adopt unsupervised learning approaches to detect insiders by analyzing the audit data that record information about employees' activities. However, in practice, we do observe a small number of insiders. How to make full use of these few observed insiders to improve a classifier for insider threat detection is a key challenge. In this work, we propose a novel framework combining the idea of self-supervised pre-training and metric-based few-shot learning to detect insiders. Experimental results on insider threat datasets demonstrate that our model outperforms the existing anomaly detection approaches by only using a few insiders.
Unsupervised Cyberbullying Detection via Time-Informed Gaussian Mixture Model
Authors: Lu Cheng, Kai Shu, Siqi Wu, Yasin N. Silva, Deborah L. Hall, Huan Liu
Venue: Conference CIKM 2020
Year: 2020

Social media is a vital means for information-sharing due to its easy access, low cost, and fast dissemination characteristics. However, increases in social media usage have corresponded with a rise in the prevalence of cyberbullying. Most existing cyberbullying detection methods are supervised and, thus, have two key drawbacks: (1) The data labeling process is often time-consuming and labor-intensive; (2) Current labeling guidelines may not be generalized to future instances because of different language usage and evolving social networks. To address these limitations, this work introduces a principled approach for unsupervised cyberbullying detection. The proposed model consists of two main components: (1) A representation learning network that encodes the social media session by exploiting multi-modal features, e.g., text, network, and time. (2) A multi-task learning network that simultaneously fits the comment inter-arrival times and estimates the bullying likelihood based on a Gaussian Mixture Model. The proposed model jointly optimizes the parameters of both components to overcome the shortcomings of decoupled training. Our core contribution is an unsupervised cyberbullying detection model that not only experimentally outperforms the state-of-the-art unsupervised models, but also achieves competitive performance compared to supervised models.
Temporal Properties of Cyberbullying on Instagram
Authors: Aabhaas Gupta, Wenxi Yang, Divya Sivakumar, Yasin N. Silva, Deborah L. Hall, Maria Camila Nardini Barioni
Venue: Conference WWW 2020
Year: 2020

Concurrent with the growth and widespread use of social networking platforms has been a rise in the prevalence of cyberbullying and cyberharassment, particularly among youth. Although cyberbullying is frequently defined as hostile communication or interactions that occur repetitively via electronic media, little is known about the temporal aspects of cyberbullying on social media, such as how the number, frequency, and timing of posts may vary systematically between cyberbullying and non-cyberbullying social media sessions. In this paper, we aim to contribute to the understanding of temporal properties of cyberbullying through the analysis of Instagram data. That is, the paper presents key temporal characteristics of cyberbullying and trends obtained from descriptive and burst analysis tasks. Our results have the potential to inform the development of more effective cyberbullying detection models.
Learning like human annotators: Cyberbullying detection in lengthy social media sessions
Authors: Peiling Yi, Arkaitz Zubiaga
Venue: Conference WWW 2023
Year: 2023

The inherent characteristic of cyberbullying of being a recurrent attitude calls for the investigation of the problem by looking at social media sessions as a whole, beyond just isolated social media posts. However, the lengthy nature of social media sessions challenges the applicability and performance of session-based cyberbullying detection models. This is especially true when one aims to use state-of-the-art Transformer-based pre-trained language models, which only take inputs of a limited length. In this paper, we address this limitation of transformer models by proposing a conceptually intuitive framework called LS-CB, which enables cyberbullying detection from lengthy social media sessions. LS-CB relies on the intuition that we can effectively aggregate the predictions made by transformer models on smaller sliding windows extracted from lengthy social media sessions, leading to an overall improved performance. Our extensive experiments with six transformer models on two session-based datasets show that LS-CB consistently outperforms three types of competitive baselines including state-of-the-art cyberbullying detection models. In addition, we conduct a set of qualitative analyses to validate the hypotheses that cyberbullying incidents can be detected through aggregated analysis of smaller chunks derived from lengthy social media sessions (H1), and that cyberbullying incidents can occur at different points of the session (H2), hence positing that frequently used text truncation strategies are suboptimal compared to relying on holistic views of sessions. Our research in turn opens an avenue for fine-grained cyberbullying detection within sessions in future work.
Truth Inference at Scale: A Bayesian Model for Adjudicating Highly Redundant Crowd Annotations
Authors: Yuan Li, Benjamin I. P. Rubinstein, Trevor Cohn
Venue: Conference WWW 2019
Year: 2019

Crowd-sourcing is a cheap and popular means of creating training and evaluation datasets for machine learning, however it poses the problem of `truth inference', as individual workers cannot be wholly trusted to provide reliable annotations. Research into models of annotation aggregation attempts to infer a latent `true' annotation, which has been shown to improve the utility of crowd-sourced data. However, existing techniques beat simple baselines only in low redundancy settings, where the number of annotations per instance is low ($\le 3$), or in situations where workers are unreliable and produce low quality annotations (e.g., through spamming, random, or adversarial behaviours.) As we show, datasets produced by crowd-sourcing are often not of this type: the data is highly redundantly annotated ($\ge 5$ annotations per instance), and the vast majority of workers produce high quality outputs. In these settings, the majority vote heuristic performs very well, and most truth inference models underperform this simple baseline. We propose a novel technique, based on a Bayesian graphical model with conjugate priors, and simple iterative expectation-maximisation inference. Our technique produces competitive performance to the state-of-the-art benchmark methods, and is the only method that significantly outperforms the majority vote heuristic at one-sided level 0.025, shown by significance tests. Moreover, our technique is simple, is implemented in only 50 lines of code, and trains in seconds.