Measuring the Effect of Influential Messages on Varying Personas (2023)

Abstract: Predicting how a user responds to news events enables important applications such as allowing intelligent agents or content producers to estimate the effect on different communities and revise unreleased messages to prevent unexpected bad outcomes such as social conflict and moral injury. We present a new task, Response Forecasting on Personas for News Media, to estimate the response a persona (characterizing an individual or a group) might have upon seeing a news message. Compared to the previous efforts which only predict generic comments to news, the proposed task not only introduces personalization in the modeling but also predicts the sentiment polarity and intensity of each response. This enables more accurate and comprehensive inference on the mental state of the persona. Meanwhile, the generated sentiment dimensions make the evaluation and application more reliable. We create the first benchmark dataset, which consists of 13,357 responses to 3,847 news headlines from Twitter. We further evaluate the SOTA neural language models with our dataset. The empirical results suggest that the included persona attributes are helpful for the performance of all response dimensions. Our analysis shows that the best-performing models are capable of predicting responses that are consistent with the personas, and as a byproduct, the task formulation also enables many interesting applications in the analysis of social network groups and their opinions, such as the discovery of extreme opinion groups.

## introduction to prevent the flooding of misinformation and hate speech on the internet, a great amount of progress has been made toward identifying and filtering such content on social media using machine learning code repository: https://github.com/chenkaisun/ response_forecasting ## dataset collection in this section, we describe how we construct data from twitter. specifically, we used twitter api foot_0 to crawl news headlines and comments below each headline from cnn breaking news foot_1 , which is one of the most popular news accounts on twitter. preprocess. we collected news headlines and corresponding comments from cnn breaking news between january 2017 and january 2019 and removed the comments that are over 50 tokens to avoid spamming. we stripped away html syntax tokens and normalized user reference with special tokens "@user". ## conclusions and future work we propose response forecasting on personas for news media, a new task that tests the model's capability of estimating the responses from different personas. the task enables important applications such as estimating the effect of unreleased messages on different communities as an additional layer of defense against unsafe information (e.g., information that might cause conflict or moral injury). we also create the first dataset for evaluating this new task and present an evaluation of the stateof-the-art neural models. the empirical results show that the best-performing models are able to predict responses with reasonable accuracy and produce outputs that are consistent with the personas. the analysis shows that the models are also able to generate contrasting opinions when conditioned on contrasting personas, demonstrating the feasibility of applying the models to discovering social groups with different opinions on issues for future work. in addition to this, an intriguing avenue for further research lies in utilizing response forecasting techniques to predict the popularity of discussion threads, as explored in previous studies @xcite .