The PGNSC Benchmark: How Do We Predict Where Information Spreads? (2024)

Abstract: Social networks have become ideal vehicles for news dissemination because posted content is easily able to reach users beyond a news outlet’s direct audience. Understanding how information is transmitted among communities of users is a critical step towards understanding the impact social networks have on real-world events. Two significant barriers in this vein of work are identifying user clusters and meaningfully characterizing these communities. Thus, we propose the PGNSC benchmark, which builds information pathways based on the audiences of influential news sources and uses their content to characterize the communities. We present methods of aggregating these news-source-centric communities and for constructing the community feature representations that are used sequentially to construct information pathway prediction pipelines. Lastly, we perform extensive experiments to demonstrate the performance of baseline pipeline constructions and to highlight the possibilities for future work.

## introduction social media platforms have become a crucial part of the information dissemination ecosystem. by allowing users to choose whom they share their socalled "content feeds" with, these platforms have created an environment in which the reach of information is amplified. pew @xcite reported that roughly half of american adults regularly consume news through social media and that 13% prefer to get their news through social media, which increases to 33% for adults under 30. further surveying suggests that adults under 30 place as much trust in news gathered from social media as from †corresponding author: alexander k. taylor {atay-lor2@cs.ucla.edu} traditional news outlets @xcite . because of this, traditionally "offline" news sources now have dedicated social media accounts that seek to propagate their content to the growing demographic of adults using social media for news consumption. given the relevance of this problem, we seek to establish a benchmark that will establish a foundation which future work may use to develop methods to identify the audiences of influential news sources and predict the flow of information. because social media users rarely explicitly disclose which news sources they get information from, we must identify the audience, or community, of each news source we consider. we will focus on two benefits of aggregating users into communities @xcite . the first benefit of community aggregation is that it allows us to leverage the structure present in the data to predict the behavior of a set of closely connected users @xcite . using this principle, we are able to circumvent the lowresource setting of user-level prediction caused by inconsistent user posting schedules by predicting the behavior of communities of users as demonstrated in prior work @xcite . the second benefit is that we can leverage inductive bias to improve the representation of the community; for instance, members of communities centered around a given news outlet will likely share that outlet's political leaning @xcite and are likely to have their views influenced by that outlet's content. this motivates the construction of community representations that incorporate as much additional information as possible (to inform community representation). we seek to take the first step in establishing a benchmark for information pathway prediction at the community level, which we title prominent global news sources for covid-19, hereafter referred to as pgnsc. pgnsc is a novel, humanvalidated dataset built using the most influential global news sources for covid-19 (pgnsc). the dataset consists of instances of news articles being posted to social media and the resulting interactions between news organization communities. we also provide a general sequential framework for building pipelines to perform information pathway prediction and define baseline methods for each pipeline stage. for the community aggregation stage, we include several methods of aggregating communities based on prior work @xcite @xcite and show their impact on information pathway prediction performance. to construct community feature representations, we seek to leverage the recent advances in large language models (llms) and their use in enhancing graph representations by using llms to summarize and encode of each organization's content @xcite . because news content often includes images, we also incorporate a jointly-trained image-text encoder into the set of community node feature generation pipelines @xcite . the appeal of pgnsc goes beyond providing data that can be used for analysis of patterns of information propagation as well as graph representation tasks. to the best of our knowledge, this is the first work to establish benchmark for information pathway prediction using heterogeneous graphs and and to use sota llms to enhance node feature representations to improve predicitons. we believe pgnsc is well positioned to serve as a vehicle for exploring how llms and graph data can be used in concert to make predictions. ## dynamic graph benchmarks the development of graph representations has recently undergone a renaissance with the development and application of graph neural networks that benefit from data richness and complexity @xcite @xcite . this evolution has underscored the critical need for robust benchmarks in both static and dynamic graph domains, divided into real-world and synthetic datasets. our focus herein is on dynamic graphs, which are pivotal for modeling time-evolving relationships in numerous applications. ## pgnsc benchmark in this section, we discuss the construction of the pgnsc benchmark. to our knowledge, this is the first publicly available benchmark in which multimodal information cascades (structure, text, and image data) have been aggregated into communitybased information pathways. ## feature initialization pipelines in this section, we describe the pipelines used to generate node feature embeddings for each community. we explore the efficacy of three approaches to pipeline construction: single-stage, two-stage, and zero-shot. ## experimental results in this section, we discuss the logistical settings of our experiments and discuss the performance of our baseline models on pgnsc. ## conclusion in this work, we present a novel benchmark for dynamic link prediction on heterogeneous graphs in the novel application domain of information pathway prediction. we consider 5 community aggregation heuristics and 7 different feature initialization pipeline constructions, all of which show the improvement that llm-enhanced feature representations can have on downstram link prediction. we also show that the community construction method significantly impacts overall performance, and our results support prior assertions of the benefits of considering interaction network structure @xcite . it is our hope that pgnsc will make llmenhanced graph tasks more accessible, fill the necessity for a large dynamic heterogeneous graph dataset, and last, but not least, offer a challenging setting for incorporating community aggregation into information propagation prediction tasks.