Stance Prediction from Multimodal Social Media Data (2023)

Abstract: Stance prediction - the computational task of inferring attitudes towards a given target topic of interest - relies heavily on text data provided by social media or similar sources, but it may also benefit from non-text information such as demographics (e.g., users’ gender, age, etc.), network structure (e.g., friends, followers, etc.), interactions (e.g., mentions, replies, etc.) and other non-text properties (e.g., time information, etc.). However, so-called hybrid (or in some cases multimodal) approaches to stance prediction have only been developed for a small set of target languages, and often making use of count-based text models (e.g., bag-of-words) and time-honoured classification methods (e.g., support vector machines). As a means to further research in the field, in this work we introduce a number of text- and non-text models for stance prediction in the Portuguese language, which make use of more recent methods based on BERT and an ensemble architecture, and ask whether a BERT stance classifier may be enhanced with different kinds of network-related information.

## introduction standard stance prediction concerns the inference of for/against attitudes towards a target topic of interest from text data. the task may be seen as analogous to sentiment (e.g., positive or negative) analysis, but stance and sentiment do not necessarily correlate @xcite . for instance, consider the following statement: in this example, given the intended target 'vaccination', the statement suggests a favourable stance. still, the overall sentiment (particularly in the use of the word 'banned') may be regarded as being more on the negative side. computational models of stance prediction, which often take social media text as an input, have been applied to a wide range of topics, including moral or social issues @xcite , politics @xcite @xcite , and others. the task has become particularly popular in the field since the semeval-2016 shared task @xcite and accompanying corpus, focusing on stance prediction from twitter text in the english language. in addition to using text data @xcite @xcite , recent work in stance prediction has addressed the use of non-text data as well @xcite . studies of this kind are largely motivated by the notion of homophily @xcite , that is, the concept of 'similarity breeds connection', and take into account a range of well-known nonlinguistic stance predictors. these include, for instance, the use of demographics information (e.g., users' gender, age, etc.) @xcite , network structure (e.g., social media friends, followers, etc.) @xcite , interactions (e.g., mentions, replies, retweets, etc.) @xcite , and other network properties (e.g., number of replies, time information, etc.) @xcite , which are often combined with standard text models. hybrid models of this kind, although not necessarily using images, audio or other media, will be hereby called multimodal. existing work in stance prediction based on hybrid data are often derived from two main nlp initiatives: (1) the iberaval-2017 shared task @xcite , devoted to the catalan and spanish languages, and (2) the evalita-2020 shared task @xcite for the italian language, in both cases providing social media corpora and accompanying non-text data. the iberaval-2017 corpus is however limited to text and gender information, whereas evalita-2020 is a truly hybrid corpus that provides both text and network-related information. similar hybrid resources (most notably for english and a few other european languages) do exist @xcite . however, we are not aware of any study in stance prediction based on hybrid data that has been devoted to our target language -portuguese. in addition to the language gap, we notice that existing work in stance prediction based on hybrid data often relies on text representations based on feature counts (e.g., bag-of-words), in many cases combined with support vector machine (svm) or other similarly time-honoured classification methods. as discussed in @xcite , some of these choices may be explained by the challenges involved in combining large text representations with (usually) much smaller non-text models (e.g., representing interactions, demographics, etc.). based on these observations, in this work we investigate whether stance prediction using a more contemporary text representation -namely, built from bert @xcite -may be enhanced with different kinds of network-related information. using a large multimodal stance corpus in the portuguese language as an input, we envisaged a number of experiments to assess stance prediction models in an ensemble architecture of textand network-related classifiers. the contributions made in this work are as follows: the remainder of this article is structured as follows. section 2 reviews existing work in stance prediction and related resources. section 3 describes our current experiment setting by presenting the models under consideration and the corpus to be taken as train and test data. section 4 presents the results of our experiments. finally, section 5 presents our final remarks and future opportunities of investigation. ## related work generally speaking, existing work in the field is largely based on twitter, a preference that may be motivated by the ease of access to text and non-text data through the twitter api. all of the selected studies are devoted to english or other european languages. among these, there are several studies focuses on catalan and spanish, including the work in @xcite , which has been developed in the light of the iberaval-2017 shared task @xcite , and which obtained the overall best results among the participant systems. similarly, several recent studies have focused on the italian language, including @xcite , which was the overall best-performing system at the evalita-2020 task b (contextual stance detection) shared task @xcite . although presently not shown, we notice also that language and topic choices usually come handin-hand, that is, existing stance datasets tend to favour target topics that are of interest to a rather local audience. this trend has been observed since the semeval-2016 english corpus @xcite , which includes us-specific topics such as trump and hillary clinton among more general topics such as climate change, etc. similarly, the catalan/spanish study in @xcite focuses on stances towards the catalan independence movement; the arabic study in @xcite addresses the issue of stance towards saudi/egyptian islands ownership; the danish cor- ## stance prediction based on hybrid data the main objective of the present work is to investigate which combinations of network-related information, if any, may be added to an otherwise standard text-based model to improve stance classification results in the portuguese language. as in the work described in @xcite , which is devoted to the italian language, we will also focus on twitter data, and on the use of friends, followers and mentions (e.g., users with whom they interact on twitter) information. however, instead of resorting to bert class probabilities as learning features, we shall make use of a full text representa-tion built from bert, and will investigate a range of simple ensemble approaches to combine (large) text and (comparatively small) non-text feature sets as a single model. ## evaluation results show that simply using all available information, as provided by the full model t+fr+fo+me, is not the best strategy at all. on the contrary, it is the use of text, friends and mentions information alone, as provided by t+fr+me, that actually delivers best results across most topics, although clearly the advantage over the second best alternative may in some cases be minimal. in order to verify the possible overall advantage of t+fr+me, this and the other top-performing model for each topic were compared by using a mcnemar test. in doing so, the advantage afforded by t+fr+me over the selected alternative strategy across topics was found to be statistically significant as follows: lula (χ = 9.8 , @xmath27.01), bolsonaro (χ = 3.0 , @xmath28.05), hydroxychloroquine (χ = 8.0 , @xmath29.05), sinovac (χ = 4.0 , @xmath30.001), church (not significant), and globo tv (χ = 21.0, @xmath31.001). this outcome further suggests a general preference of t+fr+me over the alternatives. ## final remarks this paper has addressed the issue of stance prediction by reporting a number of experiments that combine text data with network-related information -represented by twitter friends, followers and mentions -in a voting ensemble architecture. results show that the use of friends and mentions, but not followers, obtained overall best results for the present setting. the present work leaves a number of opportunities for further improvement. first, we notice that the present ensemble architecture is limited to the use of a simple majority voting method, and that more sophisticated strategies may increase overall model accuracy. this may be the case, for instance, of stacking (wolpert, 1992), and others. second, the current bag-of-users approachwhich has been taken as the basis for the present network-related models -may be replaced for a dense network representation provided by node embeddings. models of this kind, which may be computed from, e.g., node2vec @xcite , would make the current friends, followers and mentions models more informative (or less sparse), and this may have a positive impact on the current results. regarding the text portion of the model, the current task may benefit from multiple, well-known nlp methods and applications. among these, we may consider the use of hate speech detection methods @xcite @xcite , authorship attribution (custódio and paraboni, 2021; @xcite , or author profiling @xcite . the latter, comprising the computational task of determining individuals' demographics from text, may help determine their stance towards a particular topic by taking into account, for instance, information regarding their political orientation @xcite , personality traits @xcite , moral values (dos santos and paraboni, 2019; pavan et al., 2023), and others. finally, it is worth noting that, for simplicity, our task definition has been presently limited to binary (for/against) stance classification. in more realistic settings, however, it would be arguably useful to consider the intermediate (or neutral) class as well. this possible extension is also left as a suggestion of future work.