Neural Temporal Opinion Modelling for Opinion Prediction on T witter (2020)

Abstract: Opinion prediction on Twitter is challenging due to the transient nature of tweet content and neighbourhood context. In this paper, we model users’ tweet posting behaviour as a temporal point process to jointly predict the posting time and the stance label of the next tweet given a user’s historical tweet sequence and tweets posted by their neighbours. We design a topic-driven attention mechanism to capture the dynamic topic shifts in the neighbourhood context. Experimental results show that the proposed model predicts both the posting time and the stance labels of future tweets more accurately compared to a number of competitive baselines.

## introduction social media platforms allow users to express their opinions online towards various subject matters. despite much progress in sentiment analysis in social media, the prediction of opinions, however, remains challenging. opinion formation is a complex process. an individual's opinion could be influenced by their own prior belief, their social circles and external factors. existing studies often assume that socially connected users hold similar opinions. social network information is integrated with user representations via weighted links and encoded using neural networks with attentions or more recently graphical convolutional networks (gcns) @xcite . this strand of work, including @xcite @xcite , leverages both the chronological tweet sequence and social networks to predict users' opinions. the majority of previous work requires a manual segmentation of a tweet sequence into equallyspaced intervals based on either tweet counts or * corresponding author time duration. models trained on the current interval are used to predict users' opinions in the next interval. however, we argue that such a manual segmentation may not be appropriate since users post tweets at different frequency. also, the time interval between two consecutively published tweets by a user is important to study the underlying opinion dynamics system and hence should be treated as a random variable. inspired by the multivariate hawkes process @xcite , we propose to model a user's posting behaviour by a temporal point process that when user u posts a tweet d at time t, they need to decide on whether they want to post a new topic/opinion, or post a topic/opinion influenced by past tweets either posted by other users or by themselves. we thus propose a neural temporal opinion model to jointly predict the time when the new post will be published and its associated stance. instead of using the fixed formulation of the multivariate hawkes process, the intensity function of the point process is automatically learned by a gated recurrent neural network. in addition, one's neighbourhood context and the topics of their previously published tweets are also taken into account for the prediction of both the posting time and stance of the next tweet. to the best of our knowledge, this is the first work to exploit the temporal point process for opinion prediction on twitter. experimental results on the two twitter datasets relating to brexit and us general election show that our proposed model outperforms existing approaches on both stance and posting time prediction. ## methodology we present in figure 1 consists of user's own tweet x i , bag-of-word representation x b i , time interval τ i between the i -1 th tweet and the i th tweet, user embedding u, and neighbours' tweet queue {d i,1 , d i,2 , . . . , d i,l }. at first, a bi-lstm layer is applied to extract features from input tweets. then the neighborhood tweets are processed by a stacked bi-lstm/lstm layer for the extraction of neighborhood context, which is fed into an attention module queried by the user's own tweet h i and topic z i . the output of attention module is concatenated with tweet representation, time interval τ i , user representation u, and topic representation z i , which is encoded from x b i via a variational autoencoder (vae). finally, the combined representation is sent to a gru cell, whose hidden state participates in computing the intensity function and the softmax function, for the prediction of the posting time interval and the stance label of the next tweet. in the following, we elaborate the model in more details: tweet representation: words in tweets are mapped to pre-trained word embeddings @xcite foot_0 , which is specially trained for tweets. then bi-lstm is used to generate the tweet representation. ## related work the prediction of real-time stances on social media is challenging, partly caused by the diversity and fickleness of users @xcite . a line of work mitigated the problem by taking into account the homophily that users are similar to their friends @xcite . for example, @xcite gauged a user's opinion as an aggregated stance of their neighborhood users. @xcite took a step further by exploiting the extracted topics, which discern a user's focus on neighborhood tweets. recent advances in this strand also include the application of gcns, with which the social relationships are leveraged to enrich the user representations @xcite . on the other hand, several work has utilized the chronological order of tweets. @xcite presented an opinion tracker that predicts a stance every time a user publishes a tweet, whereas @xcite ) extended the previous work by introducing a topic-dependent attention. @xcite considered diverse social behaviors and jointly forecast them through a hierarchical neural network. however, the aforementioned work requires a manual segmentation of a tweet sequence. furthermore, they are unable to predict when a user will next publish a tweet and what its associated stance is. these problems can be addressed using the hawkes process @xcite , which has been successfully applied to event tracking @xcite , rumor detection @xcite @xcite and retweet prediction @xcite . a combination of the hawkes process with recurrent neural networks, called recurrent marked temporal pointed process (rmtpp), was proposed to automatically capture the influence of the past events on future events, which shows promising results on geolocation prediction @xcite . benefiting from the flexibility and scalability of neural networks, several work has been done in this vein including event sequence prediction @xcite and failure prediction @xcite . our work is partly inspired by rmtpp, but departs from the previous work by jointly considering users' social relations and topical attentions for stance prediction on social media. ## conclusion in this paper, we propose a novel neural temporal opinion model (ntom) to address users' changing interest and dynamic social context. we model users' tweet posting behaviour based on a temporal point process for the joint prediction of the posting time and stance label of the next tweet. experimental results verify the effectiveness of the model. furthermore, visualisation of the topics and attention signals shows that ntom captures the dynamics in the focused topics and contextual attention.