One-Model-Connects-All: A Unified Graph Pre-Training Model for Online Community Modeling (2023)

Abstract: Online community is composed of communities, users, and user-generated textual content, with rich information that can help us solve social problems. Previous research hasn’t fully utilized these three components and the relationship among them. What’s more, they can’t adapt to a wide range of downstream tasks. To solve these problems, we focus on a framework that simultaneously considers communities, users, and texts. And it can easily connect with a variety of downstream tasks related to social media. Specifically, we use a ternary heterogeneous graph to model online communities. Text reconstruction and edge generation are used to learn structural and semantic knowledge among communities, users, and texts. By leveraging this pre-trained model, we achieve promising results across multiple downstream tasks, such as violation detection, sentiment analysis, and community recommendation. Our exploration will improve online community modeling.

## introduction an online community refers to a group of individuals who share similar interests or purposes and utilize the internet as a means to communicate with one another @xcite . it serves as a vital component of social networks, including platforms like nextdoor and facebook groups. typically, an online community is composed of users, user-generated textual content, and communities. members within the same community often share common languages, behavior rules, and social cognition, and actively contribute to the community by posting or commenting on various *corresponding author topics and discussions @xcite . there are many studies focused on integrating online community to solve social problems such as abusement detection @xcite , conflict prediction @xcite , and bias study @xcite . previous research mainly aims at a specific social problem. although they achieve positive results on target tasks, their methods require designing different models for different tasks, which is very costly. the training process also requires high-quality, annotated datasets, which is very time-consuming and labor-intensive. in addition, there is a strong connection among communities, users, and texts as shown in figure 1 . but previous research did not take into account all these three components simultaneously. some work only modeled the relationship between communities and users @xcite @xcite , while others only utilize the community and their textual content @xcite . what's more, these studies model the online communities' relationships in a simple way such as appending community information to the text as an additional contextual input @xcite . in this paper, we target a unified pre-training framework that simultaneously models community, textual content, and user along with their information flow, such that one pre-trained model can support a wide range of downstream tasks for online community. first, to achieve this, we begin by constructing a heterogeneous graph that captures the complex relationships among communities, users, and textual content. the constructed graph consists of three types of nodes (community, user, and text) and three types of edges (post in, post by and active). this facilitates the subsequent pre-training tasks can capture community-level, user-level, and text-level information and the connection among different levels. next, we introduce two self-supervised tasks such that the pre-trained model built upon the constructed heterogeneous graph can encode rich knowledge of communities, users, and texts. text reconstruction task is designed to enable the model to learn the language style of communities and users, while the edge generation task aims to focus on the connection among texts, communities, and users. third, to support different downstream tasks within online community, we fine-tune the pretrained model on various downstream tasks such as text classification and community recommendation. by simply adding a fully connected layer, our model can be easily applied to communitylevel, user-level, and text-level tasks. extensive experiments have been completed to confirm the effectiveness of our framework. our main contributions are as follows: firstly, we propose a unified framework containing communities, users, and textual content simultaneously. secondly, we employ graph-based approaches to model the relationship among these three elements. lastly, we use two self-supervised tasks to pretrain this ternary graph, making it easy to adapt to multiple downstream tasks related to social media. ## proposed framework the proposed framework, as illustrated in figure 2 , consists of three components: graph construction, pre-training, and downstream tasks. ## conclusion in this paper, we propose a unified graph pretraining model for online community modeling. it contains communities, users, and textual content simultaneously and can be easily used in various downstream tasks related to social media. extensive experimental results validate the effectiveness and generalization capability of our approach.