Polarization identification on multiple timescale using representation learning on temporal graphs in Eulerian description
Abstract
Social media is often described as both reflecting and distorting real-life debates.
Indeed, social division occurs not only offline but also online on various political topics.
Several studies propose tools to identify and quantify online controversies through stance detection or polarization measures.
While polarization is typically studied as a “snapshot" in time of a social network, we consider it as a temporal process.
Moreover, the recent evolution in temporal graph representation learning provides new tools to directly combine time, content and graph topology.
Current techniques that characterize polarization are beginning to use these tools, typically using a Lagrangian description which focuses on user trajectories.
In this article, we make a case for approaching these problems with a Eulerian description, the concurrent description in fluid mechanics.
In this description, the temporal evolution of nodes embeddings is represented with a deformation of velocity vector fields.
Finally, we validate our method on a retweet graph from the last French presidential election campaign.
Introduction
The COVID-19 crisis shed light on scientific controversies in real-time.
Among them were the effectiveness of masks and drugs such as hydroxychloroquine, or the risk-benefit balance of vaccination.
Unsurprisingly, these controversies quickly became part of public discourse, sometimes leading to public health issues.
As a result, the general public re-discovered the fact that research and politics or media discourse are on different time scales.
Political and media expectations have contributed to a politicisation of some scientific controversies for which scientific consensus was being built at the time.
The consequence is a polarisation of the subject along pre-existing political dividing lines.
For example, in the United States, Republicans and Democrats appear to greatly differ in their approaches towards vaccination against COVID-
(Fridman et al.
These dividing lines also appear online where evolution may be measured.
There exists a direct link between opinion dynamics and the study of social dividing lines, as a shift of these lines indicates that users switch sides.
Polarization is usually evaluated on static networks, once communities are already divided.
However, this approach does not characterize the process under which this polarization arises.
Instead, using temporal graphs, we treat polarization as an evolution of social community boundaries inside the network, to detect it before being too late.
This paper develops an approach based on representation learning of temporal graphs.
Like recent works, the goal is to embed graph evolution inside a latent space to study community evolution (Li et al.
Our work is based on an implementation of Temporal Graph Networks (Rossi et al.
[2020]) to embed retweet graphs.
However, instead of using a Lagrangian description, i.e. following trajectories of users, polarization is then seen through a Eulerian description from fluid mechanics and represented with "velocity" vector fields, focusing on specific locations in the latent space.
Related works 2.1 Measure polarization
Many measures of polarization have been proposed, most of them coming from models of opinion dynamics (Gurukar et al.
[2020], Baumann et al.
[2020]) but not always suited for graph topology.
A large literature studies stance detection to approach polarization.
It can be based on content (tweets or hashtags), interactions (follows, retweets, etc) or users directly (name, location, bio, etc).
For Twitter, interaction-based solutions generally provide better results (Magdy et al.
If supervised approaches exist (Borge-Holthoefer et al.
[2014], Kutlu et al.
[2018]), more recent solutions are unsupervised using, for instance, force-directed layouts (Darwish et al.
Lai et al.
[2019] work on dynamic stance detection on various interaction graphs, exhibiting the importance of retweets.
Polarization can also be measured directly on the graph, using boundaries between communities for instance (Guerra et al.
Another solution from Morales et al.
[2015] defines a metric based on the relative sizes of detected communities and distances between their centroids.
Solutions based on embeddings are now vastly used (Rashed et al.
but they are mostly limited to static graphs.
Finally, Raghavendra et al.
[2022] models the evolution of signed networks with a dynamic Graph Neural Network (GNN), these networks being well suited for societal applications such as polarization measurement (Bonchi et al.
So far, solutions based on polarization evolution (Kearney [2019]) (opposed here to polarization as a static situation) do not fully benefit from more recent development in representation learning on dynamic graphs.
This paper aims to reconcile the study of polarization evolution with recent representation learning techniques by creating dynamic embeddings and computing their deformation.
2.2 Eulerian approach
Each new interaction changes nodes embeddings.
By gathering these changes, we can model embeddings deformations.
Velocity vector fields are used to represent them.
Instead of following users or communities along the time and trying to predict their trajectories, we focus on areas of the latent space and computing the direction of users passing through areas without following them.
This is inspired from the dichotomy between Lagrangian and Eulerian descriptions in fluid mechanics.
Eulerian descriptions of complex systems have been used in the study of road traffic or crowd flow (van Wageningen-Kessels et al.
[2016], Ma et al.
To the best of our knowledge, the most recent Eulerian approaches for social network are from Mirtabatabaei et al.
[2014], Canuto et al.
This approach attempts to overcome Lagrangian limits of opinion dynamics models such as the Hegselmann-Kraus model.
Due to the Eulerian description, they manage to prove properties of the models, reusing ideas from fluid mechanics with operators such as divergence for instance.
This changing paradigm focuses on flows rather than objects, exhibiting almost instantly fault lines compare to Lagrangian solutions .
2.3 Representation learning on dynamic graphs
As presented by Kazemi et al.
[2020] in their survey on representation learning for dynamic graphs, existing models can be seen as an encoder/decoder scheme.
The encoder associates nodes to their embeddings, from which the decoder performs a specific task, such as link prediction or node classification.
As stated in Rossi et al.
[2020], deep learning models on dynamic graphs can be classified according to the temporal dimension representation, discrete or continuous.
The discrete time setting poses several difficulties: limited temporal granularity, lost order of appearance of edges, and impossible identification of links that frequently appear together (Skarding et al.
Different approaches have thus been proposed to model dynamic graphs in a continuous setting.
Nguyen et al.
[2018] and Bastas et al.
[2019] build network representation learning methods using temporal random walks to generate structure and continuous time-aware embeddings.
ATiSE (Xu et al.
[2019]) leverages time series method for temporal knowledge graph embedding model.
Specific dynamic graph neural networks architectures have also been developed, such as Dyrep (Trivedi et al. [2019]), which produces dynamic low-dimensional node embeddings via a learned set of functions,
Jodie (Kumar et al.
[2019]), which uses coupled RNNs to learn node embedding trajectories, or
Temporal Graph Attention (TGAT)
(Xu et al.
[2020]), which introduces a new temporal graph attention layer allowing it to efficiently aggregate temporal and topological features.
These three architectures can be seen as special cases of general framework, Temporal Graph Networks (Rossi et al.
[2020]), that adds a memory module to TGAT.
TGN’s framework being the most generic, we choose to use it as a reference model for our study.
Method Description
As detailed in 2.3, the tool we have selected to model the evolution of online behaviors is the TGN framework * , the goal being to analyze the observable dynamics in its latent space.
We therefore start by presenting the general workflow of the algorithm.
The approach proposed by TGN to record the history of a node, i.e. the set of events in which it took part (interactions, change of features, etc.), relies on the use of a memory module.
The latter, based on a RNN, allows to update a memory vector encoding the past behaviors of the nodes.
It is coupled to an embedding module allowing to aggregate the neighborhood information and which can be realized by different functions: identity, a sum, an attention mechanism (Veliˇckovi´c et al.
[2018]), etc.
These two modules, memory and embedding, constitute an encoder associating to a dynamic graph a vector representation of its nodes.
This representation can then be used by a decoder to perform a specific task.
In this study, we will use a decoder associated to link prediction, the training being then self-supervised.
Indeed, the model learns, for each batch, to predict the probability of next interactions.
A more detailed description of the algorithm is provided in appendix A.1.
By removing the final MLP, TGN can be used to create node embeddings dynamically.
Depending on the wanted time scale, a time step is defined (from minutes to a week).
Embeddings can then be compared between two instants to understand the geometric evolution of the landscape.
As stated before, a Lagrangian description with direct reading of trajectories is not the only way to describe polarization.
In our Eulerian perspective, we compute the evolution of nodes in areas at the wanted granularity.
To do so, the solution adopted consists of defining a square mesh on the latent space * .
For each square, the average move between two time steps is evaluated.
To this end, the difference between positions of two time steps is computed and averaged over users located in the square at the beginning.
This calculation is only performed for squares whose numbers of users exceed a threshold in order to ensure a certain significance.
The move per square is represented with a vector, leading to a vector field (Figure 1), where angles are associated with varying colors.
Experiments and results 4.1 Preliminary analysis
Our usecase is based on a French political dataset from Twitter.
Experiment’s settings and data collection are detailed in Appendix A.2 and A.3.
The first step is to provide meaningful interpretations for the embedding reduced dimensions.
Interestingly, the embeddings obtained provide similar results to other representations (Ramaciotti Morales et al.
[2021], Chomel et al.
[2022]) such as force-directed algorithms leading to interpretable axis of the PCA.
For instance, on Figure 2, communities are computed using Louvain algorithm on the static graph to exhibit the political colors of users.
Extreme right parties appear on the left side (dark blue) and left parties appear in the bottom (red and pink).
Lastly, the government party is on the top right (yellow).
It seems that specific regions of the latent space can be linked to political communities and and we will use this information to link deformations to political events.
4.2 Deformation examples
The first striking result is the apparent disorganization in regions with less nodes such as the center.
In these regions, without strongly defined communities, there is no reason for close users to move in identical directions.
We will thus focus the analysis on dense regions of the Figure 2, as they provide consistent and explainable evolutions.
A first example is studied with a time step of about an hour.
It exhibits a pendulum swing that commonly occurs on the platform with an event bringing two groups closer before they take their distances again (Figure 1).
On the left panel, nodes at the bottom are going to the left.
Politically speaking, it means that users from the extreme left users are getting closer during this hour to alt right ones.
This matches perfectly the network activity as several major alt right accounts including the head of the movement, Florian
Philippot, has been largely retweeted.
In the meantime, extreme left figures have been less active, leading to the large departure from this area.
During the same time interval, the extreme right around Zemmour promoted his role in the help of a victim of aggression, gathering users of the community.
During the next time step, leaders of the extreme left, such as Jean-Luc Mélenchon, were massively retweeted when they criticized the government.
At the same time, the alt right was less active, pushing users from the bottom to the right of the plot.
Another example, at a different time scale, can be found in Appendix A.4.
Longer time scale solutions can be used to identify polarization directly.
Discussion
The method proposed in this study, based on the visual analysis of the deformations of the latent space, can be used to identify social phenomena such as polarization * .
Nevertheless, two questions appear naturally from the identification of such phenomena.
Actors/Users Which actors/users are involved and can explain these evolutions?
An in-depth analysis of the role of leaders has been provided for the use case, but can it be automated?
The aforementioned deformations correspond to a synthetic representation of the dynamics of nodes embeddings, allowing an efficient visualisation and detection of social phenomena.
Once detected, it would thus be interesting to perform a dynamic clustering on nodes embeddings instead of deformations, focused on the identified period and location.
Content/Topics Is it possible to find which topics are covered by these deformation?
The focus is now on the semantic aspect of opinion dynamics, requiring to interpret the space of nodes representations in a meaningful way.
Analogously to the opinion space used in Mirtabatabaei et al.
[2014], the idea is thus to give an intrinsic meaning and not simply relative to the positions of nodes in this space.
An approach would be to decode nodes embeddings via the decoder associated with the text encoder used to generate the node and edge features, with the hope that the information contained in these text embeddings has been preserved by TGN.
In practice, the memory of a node is the hidden vector, initially zero, of a RNN, which is updated after each event involving this node. More precisely, when a node interacts, a message storing information about this interaction is computed and used as the input vector of the RNN, with the previous memory of the node serving as the input hidden vector. The output hidden vector then becomes the new memory of the node.
In fact, as interaction data are processed in batches, it is necessary to define an aggregation rule for the messages produced within the same batch. Here, only the last computed message is retained. More formally, for an interaction involving nodes i and j , the messages m i and m j will be the following concatenations: [Equation]
where s k ( t − ) refers to the memory of node k ∈ { i, j } before its update and ∆ t k the time difference between the last update time of node k ∈ { i, j } and that of the interaction at the origin of the message. A.1.2 Embedding module The general form adopted for this module in Rossi et al. [2020] is:
[Equation]
where N li ([0 , t ]) denotes the time neighborhood at rank l of node i on period [0 , t ] , v i ( t ) and v j ( t ) denote nodes i and j features, and h a learnable function.
In the TGN configuration adopted for this study, the representation of a node i ∈ (cid:74) 1 , n ( t ) (cid:75) is obtained by applying the temporal graph attention layer developed in Xu et al.
A.1.3 Final MLP and Loss Function
The decoder is composed of an MLP followed by a sigmoid function, allowing the prediction of link probability between nodes i, j ∈ (cid:74) 1 , n ( t ) (cid:75) 2 at time t ≥ 0 via the following formula:
[Equation]
In training, the estimated probabilities of the edges of the batch being processed and of those obtained by negative sampling (noted ( p + i ) i ∈ (cid:74) 1 ,M (cid:75) and ( p − i ) i ∈ (cid:74) 1 ,M (cid:75) , where M is the batch size) are injected into a Binary Cross Entropy loss function: [Equation]
The experiments presented in this section are performed with the TGN-attn configuration of Rossi et al. [2020], i.e. nodes memory is used, the memory updater is a GRU, the embedding module is an attention layer considering the 10 last neighbors, message aggregation is done by keeping only the most recent one and identity is used as message function. The hyperparameters are the same as those used in the original paper.
The code allowing the reproduction of these experiments * is available online * . The configuration providing the better results has been selected. [Table] Before considering embeddings deformation, one must ensure that these embeddings almost converged to their true position.
As expected, during the first time steps, embeddings quickly evolve before stabilizing (or more exactly moving more slowly) and exhibiting interesting deformations. A.3 Datasets This dataset is collected through the Twitter API, by following 3000 personalities from the French political world. More precisely, all their immediate neighbors are considered, as well as the interactions that link them ( mentions , retweet , quote , etc.).
In addition, tweets and authors mentioning several hashtags related to the 2022 presidential elections are also collected. A more detailed description of this data collection is available as it has been used to work on the last two French presidential elections (Gaumont et al. Any other graph with features could also be used. From this dataset, a graph is built.
It is obtained by selecting a period and by choosing the considered interaction (defining what an edge corresponds to). In our case, a dynamic graph is thus obtained, noted Politics , spanning 431999 seconds, including 30682 nodes and 1088815 interactions. Edges features are BERT embeddings of retweeted texts (dimension 300 ). Node features are zero vectors of dimension 172 .
A second example has been run with a one day time step.
Instead of observing the impact on the network of a few tweets, this larger time step allows us to identify larger political movements.
A consistent result across the last week before the first round is the exodus from one extreme right community (Zemmour) to the other one (Le Pen). It can be identified with the flow from the left to the bottom of the space. This result is consistent with political analysis from media, polls and eventually outcomes from the French presidential election of 2022.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] See Section 4 for a use case based on the announced approach.
(b) Did you describe the limitations of your work?
[Yes] See Section 5 for a discussion of improvements that could be made.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See footnote 5 in Section 5.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See footnotein Section A.2.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Sections A.3 for datasets description and A.2 for TGN settings and hyperparameters.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See footnote 3 . 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] For TGN, see the end of Section 2.3.
(b) Did you mention the license of the assets?
[Yes] See footnote 1 in Section 3.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[Yes] The data collection was described in a reference and collected using the Twitter API.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] The collected data are tweets and retweets from French politics.
User ids are kept separately from users activities.
The user decoding is only used on nodes with highest degrees as they are public personalities.
It is used to make the political affiliation of communities.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]