Transformer-based Prediction of Emotional Reactions to Online Social Network Posts (2023)

Abstract: Emotional reactions to Online Social Network posts have recently gained importance in the study of the online ecosystem. Prior to post publication, the number of received reactions can be predicted based on either the textual content of the post or the related metadata. However, existing approaches suffer from both the lack of semantic-aware language understanding models and the limited explainability of the prediction models. To overcome these issues, we present a new transformer-based method to predict the number of emotional reactions of different types to social posts. It leverages the attention mechanism to capture arbitrary semantic textual relations neglected by prior works. Furthermore, it also provides end-users with textual explanations of the predictions. The results achieved on a large collection of Facebook posts confirm the applicability of the presented methodology.

## introduction most online social network (osn) platforms allow users to annotate posts with personal reactions. reactions to social posts not only indicate the user sentiment (e.g., like, dislike) but also reflect emotions or feelings (e.g., sadness, love, care). moreover, the quantity of reactions is also a direct measure of the popularity of a post and, indirectly, can indicate the audience's enthusiasm for a specific topic. such annotations are particularly relevant to marketers, advertisers, and policymakers because they can be exploited to profile osn users' behaviors and personalize the offer of related services/products. at large, understanding how users react to different types of content on osns is of paramount importance to studying human behavior and the online ecosystem. predicting the emotional reactions triggered by an osn post before its publication is particularly appealing as it enables ad hoc content revision and prioritization. the information available antepublication encompasses the textual content of the post and a set of related metadata (e.g., publisher, publication date, presence of links or associated images). previous approaches to emotional reaction prediction @xcite rely on traditional occurrence-based text statistics on text, e.g., tf-idf @xcite . thus, they ignore the semantics behind the text. furthermore, the prediction models are used as closed boxes and do not provide any explanations of the predicted reaction. this paper proposes a new approach to emotional reaction predictions based on transformers @xcite and shapley-based explanations. the proposed architecture encodes both the textual content of the post and the related metadata to obtain attention-based predictions of the number of reactions per type. on top of a multitask regressor, we use an established explainable ai model, namely shap (lundberg and lee, 2017), to provide end-users with explanations on the most influential textual features. the main contributions of this paper can be summarized as follows: ## related work despite the availability of various sources of data, accurately predicting content popularity on osns is still a challenging task due to their dynamic nature and the presence of various factors. several techniques and models have been proposed to address this challenge. typically the forecast objectives are popularity metrics, such as the number of likes a post will get. the features used for the prediction are obtained within the osn, such as the content of the post, the author of the post, the previous posts, and antecedent metrics on the performance of the post, or are gathered from outside of the osn, e.g., presence in newspapers and tv shows. popularity @xcite is the main sign of success on osns, and, as a consequence, it is the focus of most pieces of research. many works predict content popularity considering content intrinsic characteristics and social interaction features. in particular, natural language processing (nlp) techniques are leveraged for helping predict future popularity. the factors that impact the popularity of posts on facebook are identified in @xcite , using an empirical analysis involving multiple linear regressions, with the most important factors being the number of followers and the presence of images. similarly, @xcite highlights the characteristics related to the dynamics of content consumption in instagram, while @xcite and @xcite predict the popularity of a post by combining user and post features. the authors of @xcite predicts different reddit news popularity indicators using the comment tree and the user graph, while @xcite ) considers the early views for prediction, focusing on features related to the intrinsic attractiveness of a video and the influence from a propagation structure. fewer works in the literature deal with emotional reaction prediction. among them, @xcite simplifies the problem to a classification task. conversely, @xcite predicts facebook reaction quantity, using nlp on the text of the post and on comments and answers to the post after publication. notice that content popularity can be also forecast from other information outside of the social network (exogenous), e.g., presence in newspapers, tv shows, etc. an example is the work by @xcite that forecasts instagram and facebook influencer popularity by extracting external data from google trends and then applying financial stock-market tools such as bollinger bands. another body of literature focuses on the prediction of the temporal dynamics of post popularity. the authors of @xcite study how influencer posts attract likes and reactions and the factors for content popularity evolution. similarly, @xcite identifies temporal evolution patterns and uses those to predict the future popularity of the content using data from youtube, digg and vimeo, applying k-means clustering and simple linear forecasting technique. finally, @xcite propose a model that reproduces the popularity attraction on twitter, observing that hourly interactions decrease geometrically with time. other works focus on predicting other dimensions of popularity, and not the intensity itself. the authors of @xcite predict popularity of posts by subdividing their evolution into three key moments ("burst", "peak", and "fade") using a support vector regression technique. in a similar direction, @xcite predicts when the popularity of twitter hashtags reaches its peak, using an lstm deep learning model with topological network information, social information, and hashtag strings. differently from previous works, in this paper we focus on predicting not only a single metric, but multiple reaction types associated with specific sentiments. we keep the regression objective to make precise and fine-grained predictions, without simplifying it to a classification problem. moreover, we only use data available at publication time, hence we do not need any other information like early popularity or comment content. indeed, our approach is only based on the content and metadata of the post itself, and on the features of the creator's profile. we leverage modern transformers to capture more subtle, semantic patterns in the text and perform better predictions. ## results in this section, we show and discuss the results of our experiments. we evaluate the proposed regression models on the dataset described in section 4.1 in terms of median absolute percentage errors. then, we delve into the transformer+metadata model and investigate the most relevant words to the regression task using the shap algorithm. ## limitations our methodology provides a novel approach to predicting emotional reactions to social posts. while our proposed methodology has shown promising results, further research is necessary to address some limitations and expand the scope of our findings. firstly, our model's reliance on only textual content and metadata may limit prediction accuracy, as other factors such as user demographics or multimedia content may also impact emotional reactions. future work could explore the inclusion of additional features. secondly, this study focuses on predicting emotional reactions to posts, rather than the reasons behind them. while our method provides textual explanations, they may not fully capture the complexity of user emotional responses. lastly, our approach was evaluated on meta posts, but may not generalize well to other social networks or domains. further evaluations on different datasets are needed to confirm the applicability of our methodology across different contexts. ## ethical considerations while our study minimizes privacy concerns by solely analyzing creator post content, we acknowledge the ethical implications of our model's predictions on individuals and communities. overall, the models proposed in the present study raise ethical concerns around the potential impact of our model's predictions on content creator behavior and the distortion of social media sentiment. our model's predictions may incentivize content creators to prioritize emotional reactions over accuracy or factual content. to address this concern, responsible and ethical use of our methodology, alongside recognition of the limitations in capturing the complexity of user behavior, is necessary. finally, we observe that the dataset used in this work has been collected in a lawful and ethical environment. we used the crowdtangle platform offered by facebook, which contains only public posts of public profiles. the metadata of posts do not contain any sensitive information and only include the aggregate number of reactions they received. ## conclusions and future work the paper explored the use of transformer-based models to predict the number of reactions antepublication to a social post. it focused on predicting the reaction counts for different types of positive and negative reactions by exploring the role of text and metadata information. compared to prior ap-proaches based on traditional text representations (e.g., tf-idf), it achieved significant performance improvements thanks to the higher capability to capture the semantics behind the text. transformers perform best compared to simpler methods even in the absence of metadata. the paper also leveraged a shapley-based explainer to identify the tokens that mostly influence the prediction outcomes. the explanations meet the expectation, especially for very positive and very negative reaction types.