Capturing Human
Values during
Controversies
Tutorial
July 17, 2023 @ Copenhagen, Denmark

Kyriaki Kalimeri
ISI Foundation
Ph.D. Brain andCognitive Sciences@KyriakiKalimeri

Giovanni Da San Martino
University of Padova
Ph.D. Computer
Science@giodsm

Yelena Mejova
ISI Foundation
Ph.D. Computer
Science@YelenaMejova

Oscar Araque
Universidad Politécnica deMadridPh.D. Telecommunication
Engineering@oaraquei2Outline
Talk

Do

Human & Moral Values (Kyriaki)

MoralStrength: extracting moral
values from text (Oscar)

Propaganda, persuasion &coordinated behaviour(Giovanni)Controversy detection (Yelena)

PRTA: detecting
manipulation/persuasion
techniques (Giovanni)
RWC: network analysis forpolarization/controversy (Yelena)3Outline
Talk

Do

Slides will be posted onslideshare and tutorial website:

Download code & sample data:

https://propaganda.math.unipd.it/ic2s2-tutorial/

https://github.com/oaraque/hu
man-values-tutorial-ic2s2-2023/

4

Human and Moral
Values5

What is a Value?

Schwartz’s Values

“Values” arebroadmotivational
goals!

http://www.worldvaluessurvey.org/

Moral Values

11Personality & Values
Values could be considered a higher levelwith respect to Personality Traits.
Moral values determine how and when
dispositions and attitudes towards
interpersonal and intergroup processes
relate with our life stories and narratives.
Personality alone does not suffice to explain
our judgments.
McAdams, D., & Pals, J. (2006). A new big five: Fundamental principles
for an integrative science of personality. American Psychologist , 61 , 204

Attitudes Towards
Environmental: Climate Change
Healthcare: Vaccination
Philanthropy: Charitable Giving
Consumer Choices: Music Preferences
Ethics: AI and moral decision-making

Moral Values Detection
Psychometric Surveys
Digital Traces- Mobile Phone data (Browsing History, Location Data,
Messaging, App, Email)- Social Media Activity (Text, Music, Photos, Videos)16

Values in the Wild
Why would you wear amask during an epidemic?

https://www.theguardian.com/lifeandstyle/2021/aug/19/anti-masker-unlikely-friendship●●●

What role do values play in the formation of our opinions?
Can these values be manipulated to inﬂuence our opinions?
What societal structures may correspond to the discussions around values?
17

18

19

20

Moral Values Detection in Text
Lexicon Based Approaches
MFD - Liberals and Conservatives Rely on Diﬀerent Sets of Moral Foundations
MoralStrength - Exploiting a moral lexicon and embedding similarity for moral
foundations prediction (code: https://pypi.org/project/moralstrength/)
LibertyMFD - A Lexicon to Assess the Moral Foundation of Liberty.
eMFD - The Extended Moral Foundations Dictionary (eMFD): Development andApplications of a Crowd-Sourced Approach to Extracting Moral Intuitions from Text
Fine-tuned Language Models
MoralBERT: Detecting Moral Values in Social Discourse21

Domain Comparison of Moral Rhetoric
All Lives Matter (ALM), and Black Lives Matter (BLM) fromDavidson et al. Automated Hate Speech Detection and the
Problem of Oﬀensive Language.
Tomea can shed light on how domain-speciﬁc language
conveys morality,‘brotherhood’ has a low impact on betrayal moral the ALM
domainbut a considerably higher impact in BLM!
Liscio, E., Araque, O., Gatti, L., Constantinescu, I., Jonker, C., Kalimeri, K. and Murukannaiah, P.K.,
2023, July. What does a Text Classifier Learn about Morality? An Explainable Method for
Cross-Domain Comparison of Moral Rhetoric. In Proceedings of the 61st Annual Meeting of the
Association for Computational Linguistics (Volume 1: Long Papers) (pp. 14113-14132).
22

Lexicon Approaches

Large Language Models✔Interpretable✔Efficiency✔Domain-specific Adaptation✔Contextual Understanding✔Ability to Handle Sarcasm and Irony✔Generalization across Domains❌Limited Contextual Understanding❌Difficulty in Handling Sarcasm❌Lexicon Incompleteness❌Computational Requirements❌Lack of Transparency❌Ethical Concern

23

24

Values in the Wild→→

How can we analyze human values using
computational tools applied touser-generated & news content?
25

GCC of the follower network coloured by METIS score

26

27

28

Who is more susceptible to disinformation andconspiracy theories?
Which moral values are expressed by those who
proliferate conspiracy theories?

29

30

Moral Values used in Propaganda
Propaganda often seeks to appeal to individuals' moral values and beliefs toinﬂuence their opinions and actions.
By aligning with or distorting moral values, propaganda can gain credibility,
emotional appeal, and a sense of moral righteousness.
Propaganda may manipulate moral values by selectively presenting
information, distorting facts, or framing issues in a way that aligns with a
speciﬁc agenda.

31Moral Values, Propaganda and Controversies
Propaganda frequently plays a role in creating or intensifying
controversies by disseminating biased or misleading information.
Controversies arise when different groups or individuals holdconﬂicting viewpoints, often fueled by differing interpretations offacts and values.
Propaganda can exploit controversies by spreading disinformation,
exaggerating divisions, or demonizing opposing viewpoints, thereby
inﬂuencing public opinion and exacerbating conﬂicts.
32Moral value assessment in natural language

33

34

Tutorial notebook●

The full code for the tutorial lives here:

https://github.com/oaraque/human-values-tutorial-ic2s2-2023●

Code for moral value assessment in natural language:

MoralValues/Moral-Value-Estimation.ipynb

35

MoralStrength●

MoralStrength is a Python module that allows us to assess moral values
using the MoralStrength lexicon.

●

We can install MoralStrength through pip:

pip install moralstrength●

Available at pypi:○

https://pypi.org/project/moralstrength/

Oscar Araque, Lorenzo Gatti, Kyriaki Kalimeri, MoralStrength: Exploiting a moral lexicon and embedding similarity for moral
foundations prediction, Knowledge-Based Systems, Volume 191, 2020, 105184, ISSN 0950-7051,
https://doi.org/10.1016/j.knosys.2019.105184.

36

0. Importing libraries

371. Read Dataset●●

We use the Moral Foundations Twitter Corpus (MFTC), a dataset
composed of Twitter messages, manually annotated with moral values
This dataset contains several categories of data, corresponding to
diﬀerent campaigns or movements:○ Hurricane Sandy○ Baltimore Protest○ All Lives Matter○ Black Lives Matter (BLM)○ 2016 Presidential Election

38

Read the dataset in JSON format

39

Some basic characteristics of the data●●

Number of instances: 4,340 documents
Distribution of labels40

2. Extract the Moral Values from natural language
For each document, we want to extract the associated Moral Values. To do so,
we use the estimate_morals function:

41

MoralStrength assessments are encoded following a Likert scale

1 2 3

4

5

6

7

8 9

The encoding range is [1, 9], being a value of 5 interpreted as morally neutral.

42

We adapt the original scale using the following formula:𝑒′ = | 𝑒 − 5 |For example,
The courage to be impatient with evil and patient with people, the courage to ﬁght for social justice. @CornelWest#blacklivesmatterhas an associated Fairness score of 7.6. In this case, 𝑒′=2.6.
Another example,#CNN #Obama #Aclu #Ap #UN #BlackLivesMatter #Chicago Declare war on these racist delusional Killers Pure
Hate and Lies #FBI #DOJ
has 𝑒=1.667 and 𝑒′=3.333
43

Now, we can use the adapted values as relevance for the moral foundation.
Modelling the task as presence classiﬁcation for each of the moral
foundations.

44Results for the unsupervised prediction of moral values
45Confusion matrix for the unsupervised evaluation46

3. Supervised classiﬁcation of Moral Values●

Previously, we have shown how MoralStrength allows us to assess
morality in natural language in an unsupervised manner.

●

Now, we show how we can use these assessments to enrich text
representations in supervised settings.

47

3.1 Pre-process text (simple)●●

Uses spacy for preprocessing
MoralStrength includes a pre-loaded reduced spacy model48

3.2 Extract TF-IDF features49

3.3 Evaluate the classiﬁer

50

Results for the baseline supervised classiﬁcation
51

Confusion matrix for the baseline supervised classiﬁcation
52

The deﬁned learning model is as follows

TF-IDF
vectorizer

Classifier(SVM)

It uses just one mode of information.
53

Now, let's evaluate a more complete model, combining the
textual model with the extracted moral values
TF-IDF
vectorizer
Unifiedrepresentation

Classifier(SVM)

MoralStrength543.4 Including information from MoralStrength55

3.5 Evaluate the combined classiﬁer

56

Results for the combined supervised evaluation
57

Confusion matrix for the combined supervised evaluation
58

Conclusions●●

MoralStrength allows us to perform unsupervised moral value analysis
on textual data
○ Incorporating its own pre-processing mechanisms
This information can be used to:○ Study morality in an unsupervised manner, directly using the moral
signals○ Enrich textual representations in a machine learning system, adding
them to additional information sources (e.g., sentiment or emotion)

59

Propaganda, persuasion & coordinatedbehaviour60

Deﬁnitions: Propaganda“Communications that deliberately misrepresent symbols, appealing toemotions and prejudices and bypassing rational thought, to inﬂuence its
audience towards a speciﬁc goal”**deﬁnition re-elaborated from Institute for Propaganda Analysis (Ed.). (1938). How to Detect Propaganda. In Propaganda Analysis. Volume I of the
Publications of the Institute for Propaganda Analysis (pp. 210–218).

61

From pre-Internet Propaganda...

●Control of mass media●

Closed Borders (non-anonymous campaigns)●

Requiring massive resources

Bolsover, G., & Howard, P. (2017). Computational Propaganda and Political Big Data: Moving Toward a More Critical Research Agenda. Big Data, 5(4), 273–276.

62

...To Computational Propaganda●“The rise of the Internet […] has opened the creation and disseminationof propaganda messages, which were once the province of states andlarge institutions, to a wide variety of individuals and groups.”

Bolsover, G., & Howard, P. (2017). Computational Propaganda and Political Big Data: Moving Toward a More Critical Research Agenda. Big Data, 5(4), 273–276.

63

Computational Propaganda Cookbook● Diﬀerent technical skills needed○

Creating persuasive messages○

Disseminating the messages (using bots)○

Maximising audience reach○

Microproﬁling

64Propaganda: Document-Level Analysis●

Binary Classiﬁcation Task: “Is a document
propagandistic?” Yes, No●

Few datasets and Models available1,2●

Annotating documents might becontroversial, requires experts(expensive)●

Lacks explainability

1 Hannah Rashkin et al. Truth of varying shades: Analyzing language in fake news and political fact-checking. In EMNLP, pages 2931–2937, 2017
2 Alberto Barrón-Cedeño et al. Proppy: Organizing the news based on their propagandistic content. Inf. Process. Manag., 56(5):1849–1864, 201965

Propaganda: Document-Level Analysis●

Binary Classiﬁcation Task: “Is a document
propagandistic?” Yes, No●

Few datasets and Models available●

Annotating documents might becontroversial, requires experts(expensive)●

1,2

Distant Supervision○

If a news source is
propagandistic 🡪 each of
its articles is○

Risk of Modelling thesource instead of theconcept of Propaganda

Lacks explainability

1 Hannah Rashkin et al. Truth of varying shades: Analyzing language in fake news and political fact-checking. In EMNLP, pages 2931–2937, 2017
2 Alberto Barrón-Cedeño et al. Proppy: Organizing the news based on their propagandistic content. Inf. Process. Manag., 56(5):1849–1864, 201966

Persuasion Techniques Detection●

Propaganda is conveyed through a series of rhetorical and psychological
techniques●

The set of propaganda techniques diﬀers between scholars 1, from○○

7 of Miller2 to~70 in Wikipedia3

1 Robyn Torok. 2015. Symbiotic radicalisation strategies: Propaganda tools and neuro linguistic programming. In Proceedings of the Australian Security and Intelligence
Conference, pages 58–65, Perth, Australia.
2 Clyde R. Miller. 1939. The Techniques of Propaganda. From “How to Detect and Analyze Propaganda,” an address given at Town Hall. The Center for learning.
3 http://en.wikipedia.org/wiki/Propaganda_techniques

67Persuasion Techniques

repetition

68Persuasion Techniques

repetition

69

70

Name Calling
71

?
72

Bandwagon

73

Argotario●

A game to educate people torecognize and create fallacies. Users○○●

recognise fallacies in others’
arguments
write fallacious arguments

A byproduct of Argotario is a corpus
with 1.3k arguments annotated with
ﬁve fallacies (including ad hominem,
red herring)○in English and German
74Change my View Corpus●

Change My View: online
moderated platform forargumentation posted on reddit●

A user posts an opinion○○

other users provide their arguments
to change his/her point of view
the original poster award points tothe convincing arguments

75

Change my View Corpus●●

Moderators remove all adhominem (attack to the person)
arguments
collected 3,396 threads with
3,866 ad hominem in total

Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. Before name-calling: Dynamics and triggers of ad hominem fallacies in web
argumentation. In Proceedings of NAACL-HLT ’18, pages 386–396, New Orleans, LA, USA.

76

Propaganda Techniques Corpus (PTC)

G. Da San Martino, S. Yu, A. Barrón-Cedeño, R. Petrov, P. Nakov, "Fine-Grained Analysis of Propaganda in News Articles", in EMNLP-IJCNLP 2019, Hong Kong, China,
November 3-7, 2019.

77

PTC Corpus●

536 news articles in English from48 sources (450k words)
annotated at fragment level with18 techniques (400 man hours
of work)●

Later extended to 2049 articles
in 9 languages1 (French, German,
Italian, Polish, Russian, Greek,
Spanish, Georgian)

1 Jakub Piskorski, Nicolas Stefanovitch, Nikolaos Nikolaidis, Giovanni Da San Martino and Preslav Nakov Multilingual Multifaceted Understanding of Online Newsin Terms of Genre, Framing, and Persuasion Techniques

78

Neural Approaches to Persuasive Spans Detection●

Output at token (word) level: beginning, middle, end of a persuasive spanor not part of any span

79

Results on English Data
Span Level

Sentence Level

A Demo of the System is available at https://www.tanbih.org/prta
80Persuasion Techniques in Memes●

Most communication in social media ismultimodal, mixing textual with visual content●

SemEval 2021 task 6: 950 memes annotated with22 techniques1●

New Data (9K memes!) and a new shared task
are coming soon!2

1 Dimitar Dimitrov et al.: Detecting Propaganda Techniques in Memes. ACL/IJCNLP (1) 2021: 6603-6617
2 SemEval 2024 Task 4: https://semeval.github.io81

Disseminating messages On Social Media (using bots)●

Detecting Fake Accounts: Botometer●

Given a Twitter account, Botometer extracts○

over 1,000 features relative to the account●

Yields a classiﬁcation score called bot score: the higher the score, thegreater the likelihood that the account is controlled by software●

Drawbacks:○○

lack of reliable ground truth
malicious actors evolve to avoid detection

Yang et al. Arming the public with artiﬁcial intelligence to counter social bots. Human Behavior and Emerging Technologies, 1(1):48–61, 201982

Deep Bot Detection●

Devise a user representation based on behaviour (posting, retweeting)and sequence of tweets’ content

Cai et al. Detecting social bots by jointly modeling deep behavior and content information. In CIKM, pages 1995–1998, 201783

Detecting Coordinated Behaviour: RTBust● Detecting coordinated behaviour
instead of fake accounts● Encode and cluster retweet patterns● Discriminate between normal andinauthentic behaviour● Rationale: humans exhibit more

behavioural heterogeneity than bots

Mazza et al. RTbust: Exploiting temporal patterns for botnet detection on Twitter. In WebSci, pages 183–192, 201984

Conclusions●●●

Bot Detection is a challenging problem: most machine learningtechniques are designed for stationary and neutral environments
Coordinated behavior is not necessarily harmful
Propaganda: “Communications that deliberately manipulate the audience
to inﬂuence it towards a speciﬁc goal”○

Detection of persuasion techniques is a recent area of research■○encouraging results but still lots to do

Detecting intent is a hard task■

Coordinated Behaviour as a proxy for intent?1

1 Giovanni Da San Martino et al. “A Survey on Computational Propaganda Detection”. In Proceedings of the 29th International Joint Conference on Artiﬁcial
Intelligence, IJCAI-PRICAI ’20. Yokohama, Japan, 2020, pp. 4826–4832.

85

Prta: Detection of Persuasion Techniques in Texts
Prta (https://www.tanbih.org/prta)• continuously collects news articles○○

highlights fragments with persuasion techniques
shows aggregated statistics• analyses articles submitted by the user through○ web interface○ dedicated API (an example of usage is available here)86

Prta: Detection of Persuasion Techniques in Texts

https://www.tanbih.org/prta

87

88

89

90

91

92

93

94

Controversy Detection

95

Kiran Garimella

https://gvrkiran.github.io/polarization/

Gianmarco De
Francisci Morales
Michael Mathioudakis

Aristides Gionis@PSM
96values97

In total, 26 out of the 52 observed
episodes (or 50% of cases) saw
their country’s Regimes of theWorld score downgraded, with thevast majority of those -- 23 -descending into some form ofauthoritarianism.

https://carnegieendowment.org/2022/01/18/what-happens-when-democracies-become-perniciously-polarized-pub-86190

98Causes of polarization: cognitive dissonance“Cognitive dissonance” – psychological conﬂict resulting from incongruous beliefs andattitudes held simultaneously
Selective exposure
Klapper. “The eﬀects of mass communication.” 1960

Subjects choose to examine items that agree with their decision

Biased assimilation

Lord et al. “Biased assimilation and attitude polarization: The eﬀects of prior theories on subsequently
considered evidence.” 1979

Subjects ﬁnd consonant evidence more convincing

Free-choice
Brehm. “Postdecision changes in the desirability of alternatives.” 1956

Spreading-apart-of-alternatives

Induced compliance
Festinger and Carlsmith. “Cognitive consequences of forced compliance.” 1959

Subjects justify their decisions a-posteriori, even if they originally disagreed

@PSM
99

Causes of polarization: group bias
Social identity complexity

Roccas, S. and Brewer, M.B., 2002. Social identity complexity. Personality and Social Psychology Review.

Individuals associate themselves with social identities
race, religion, gender, classGroup polarization

Sunstein, C.R., 2002. The law of group polarization. Journal of political philosophy.

The tendency for a group to make decisions that are more extreme than the initial inclination of its
members@PSM
100

Causes of polarization: algorithmic bias
Personalizationin news, search, shopping, dating, any content
Social feedback
homophily, rich gets richer, groupthink
Filter bubble - intellectual isolation that can result frompersonalized search/recommendation
Echo chambers - environment or ecosystem in which
participants encounter beliefs that amplify or reinforce theirpreexisting beliefs by communication and repetition inside
a closed system and insulated from rebuttal
Cinelli, Matteo, et al. "The echo chamber effect on social media." PNAS. 2021

@PSM
101

Deﬁning polarization
A society can be thought of as an amalgamation of groups,
where two individuals drawn from the same group are“similar,” and from diﬀerent groups, are “diﬀerent” relative tosome given set of attributes. The polarization of a distributionof individual attributes must exhibit the following basic
features:1.
2.
3.

There must be a high degree of homogeneity withineach group
There must be a high degree of heterogeneity acrossgroupsThere must be a small number of signiﬁcantly sizedgroups. In particular, groups of insigniﬁcant size (e.g.,
isolated individuals) carry little weight.

less polarized

more polarized
moreequal!

even morepolarized@PSM
Esteban, J. M., & Ray, D. (1994). On the measurement of polarization. Econometrica: Journal of the Econometric Society, 819-851.

102

Deﬁning polarization
highly polarized

less polarized(less cohesive groups)

more polarized

Esteban, J. M., & Ray, D. (1994). On the measurement of polarization. Econometrica: Journal of the Econometric Society, 819-851.

103

Deﬁning polarization
Measures of polarization:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

spreaddispersion
coverageregionalization
community fragmentationrole of groups in some senses of polarization
distinctness
group divergencegroup consensus
size parity

Bramson, A., Grim, P., Singer, D. J., Fisher, S., Berger, W., Sack, G., & Flocken, C. (2016). Disambiguation of social polarization concepts andmeasures. The Journal of Mathematical Sociology, 40(2), 80-111.

@PSM
104

Deﬁning polarization
Measures of polarization:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

spreaddispersion
coverageregionalization
community fragmentationrole of groups in some senses of polarization
distinctness
group divergencegroup consensus
size parity

Bramson, A., Grim, P., Singer, D. J., Fisher, S., Berger, W., Sack, G., & Flocken, C. (2016). Disambiguation of social polarization concepts andmeasures. The Journal of Mathematical Sociology, 40(2), 80-111.

@PSM
105

Detecting polarization: content
Social media: hashtags as topic indicators

Inverse document frequency (ingeneral subset)#baltimoreriots

sets of Words and Hashtags that co-occur with hashtag hx#netanyahuspeech

Garimella, De Francisci Morales, Gionis & Mathioudakis. “Quantifying Controversy in Social Media.” WSDM 2016.
Klenner, Amsler, Hollenstein & Faaß. “Verb Polarity Frames: a New Resource and its Application in Target-specific Polarity Classification.” KONVENS @PSM
2014.
106

Detecting polarization: content●●

Controversy lexicons
Controversial topics have:○○○strongly biased terms
more negative terms
fewer strongly emotional terms“we show that we can indicate to what
extent an issue is controversial, bycomparing it with other issues in terms of
how they are portrayed across diﬀerent
media.”@PSM
Mejova, Zhang, Diakopoulos, and Castillo. “Controversy and Sentiment in Online News.” C+J Symposium 2014.

107

Detecting polarization: content
Controversial topic - a concept that invokes conﬂicting sentiments
Subtopic - factor that gives a particular sentiment (+ve or -ve) - noun phrases
Assumption - a controversial topic receives contrasting sentiment (of diﬀerent kind)
Controversiality - sum of magnitudes of sentiments around subtopics, and their diﬀerenceChoi, Jung & Myaeng. “Identifying controversial issues and their sub-topics in news articles.” PAW-ISI 2010.

@PSM
108

Detecting polarization: content--

Use Google’s Multilingual Universal Sentence
Encoder (MUSE) with pre-trained CNN
embeddings to represent posts
Project each user vector onto a
two-dimensional plane using Uniform Manifold
Approximation and Projection (UMAP)
algorithm
Cluster the projected user vectors using
hierarchical density based clustering (HDBSCAN)
Compare clusterings of embeddings fordiﬀerent topics

Rashed, A., Kutlu, M., Darwish, K., Elsayed, T., & Bayrak, C. (2021, May). Embeddings-based clustering for target specific stances: The case of apolarized turkey. In Proceedings of the International AAAI Conference on Web and Social Media (Vol. 15, pp. 537-548).

109

Detecting polarization: content●●

Find out if a Web page discusses a (known) controversial topic
Map topics (named entities) in a Web page to Wikipedia articles○○●

A Web page is controversial if it is similar to a controversial Wikipedia article
E.g., If a news article mentions Abortion it is controversial

Related:○○

There is a lot of work on identifying controversial topics on Wikipedia
Edit wars, hyperlink structure, etc.

Dori-Hacohen and Allan. “Detecting Controversy on the Web.” CIKM 2013.

@PSM
110

Detecting polarization: network●

Retweet network for politicalhashtags has a bi-clusteredstructure○●

Retweet network exhibits a highly
modular structure, segregating
users into two homogenous
communities corresponding to thepolitical left and rightUsers mention/reply to othersfrom their opposing viewpoint

RetweetMention@PSM
Conover, Ratkiewicz, Francisco, Gonçalves, Menczer, and Flammini. “Political Polarization on Twitter.” ICWSM 2011.

111

Detecting polarization: network●●●

Deﬁne reply trees
Identify frequency of motifs inthese trees
Take into account also social
graph (follower information)@PSM
Coletto, Garimella, Luchesse, and Gionis. “A Motif-based Approach for Identifying Controversy.” OSNEM. 2017.

112

Detecting polarization: network●●

Community boundary - possible expressions of antagonism
Boundary node:
○
○

●
●
●
●

have at least one edge that connecting to the other community
have at least one edge connecting to a member of its community which
does not link to the other community“polarized networks tend to exhibit low concentration of
popular nodes along the boundary”
P(v) = dinternal(v)/(dexternal(v) + dinternal(v)) – 0.5
P(v) > 0 → v prefers internal connections (antagonism?)
P(v) < 0 → v prefers connections with members of the othergroup (increased homophily!)

Guerra, Meira, Cardie, and Kleinberg. “A Measure of Polarization on Social Media Networks Based on Community Boundaries.”
ICWSM 2013.

@PSM
113

Detecting polarization: network
Gun debate network
Boundary polarization shows communities 2& 3 agree
GC-2

GC-3

GC-1

Guerra, Meira, Cardie, and Kleinberg. “A Measure of Polarization on Social Media Networks Based on Community Boundaries.”
ICWSM 2013.

114

Detecting polarization: network●

Opinion formation:○●

Propagation of opinions from “elite” users to “listeners”

Measure: distance between distributions○○

Distance between two gravity centers of opinions
Accounts for the mass of the population@PSM
Morales, Borondo, Losada, and Benito. “Measuring political polarization: Twitter shows the two sides of Venezuela.” Chaos. 2015.

115

Detecting polarization: network●●

Bi-partition retweet (endorsement) graph using METIS
Random Walk Controversy (RWC) Score: “Consider two random walks, one
ending in partition X and one ending in partition Y , RWC is the diﬀerence ofthe probabilities of two events: (i) both random walks started from thepartition they ended in and (ii) both random walks started in a partition otherthan the one they ended in”(i) These probabilities are not skewed by the size of each partition, as the random walk starts with equal
probability from each partition, and(ii) they are not skewed by the total degree of vertices in each partition, as the probabilities are conditionalon ending in either partition (i.e., the fraction of random walks ending in each partition is irrelevant).
Garimella, De Francisci Morales, Gionis, and Mathioudakis. “Quantifying Controversy on Social Media.” Transactions on Social Computing. 2018. 116

Detecting polarization: network
Input: information cascade (retweet) and social (follow)
networks
Probabilistic generative model with latent variables:●●θc ,u ∈ [0, 1]: the level of polarized engagement ofuser u in a echo chamber c
φc ,u ∈ [0, 1]: the level of social engagement of user
u in a social community cInferred using Generalized Expectation Maximization
algorithm

Minici, M., Cinus, F., Monti, C., Bonchi, F., & Manco, G. (2022, October). Cascade-based echo chamber detection. In
Proceedings of the 31st ACM International Conference on Information & Knowledge Management (pp. 1511-1520).

117

Detecting polarization: network
BrexitReferendum

VaxNoVax

conductance —how closely-knitted is the community with the rest of the graphpurity—the ratio of users with the same ideological alignment, measured as theaverage polarity of the tweets they reshare
Minici, M., Cinus, F., Monti, C., Bonchi, F., & Manco, G. (2022, October). Cascade-based echo chamber detection. In
Proceedings of the 31st ACM International Conference on Information & Knowledge Management (pp. 1511-1520).

118

Audience suggestion
Suggested reading during discussion:
Salloum, A., Chen, T. H. Y., & Kivelä, M. (2022). Separating polarization fromnoise: comparison and normalization of structural polarization measures.
Proceedings of the ACM on human-computer interaction, 6(CSCW1), 1-33.
https://dl.acm.org/doi/abs/10.1145/3512962

119Case Study: measuring anti-vax echo chambers
Echoes through Time: Evolution of the Italian COVID-19 Vaccination Debate
Giuseppe Crupi, Yelena Mejova, Michele Tizzani, Daniela Paolotti, Andre Panisson@ International AAAI Conference on Web and Social Media (ICWSM) 2022

●
●
●
●
●
●

Twitter Streaming API
Italian language filter
Sep 5, 2019 – Nov 7, 2021
>16M users, 665K tweets
6 time periods
6 retweet “endorsement”
retworks (weight > 1)

120

pet

Opinion communities
supporting
hesitant●hierarchical clustering +selection using modularity●

manual annotation of users●

strong separation●

pet users bridge hesitantand supporting camps

Crupi, Mejova, Tizzani, Paolotti, Panisson. Echoes through Time:
Evolution of the Italian COVID-19 Vaccination Debate. ICWSM 2022

121

Measuring echo chambers: membership●●●

tweets per user per day

almost nobody changes
sides
most vaccine
supporters: early
vaccine period (iv)most vaccine hesitant:late vaccine period (vi)hesitant

pet
Crupi, Mejova, Tizzani, Paolotti, Panisson. Echoes through Time:
Evolution of the Italian COVID-19 Vaccination Debate. ICWSM 2022

supporting122

Measuring echo chambers: endorsement● Random Walk Controversy score:○ “how likely a random user on either side

is to be exposed to authoritative content
from the opposing side”[Garimella et al. TSC’18]

Crupi, Mejova, Tizzani, Paolotti, Panisson. Echoes through Time:
Evolution of the Italian COVID-19 Vaccination Debate. ICWSM 2022
123

Measuring echo chambers: mentions●●

pre-COVID two sides
almost do not mention
each otherduring vaccine rollout,
both sides mentioned eachother almost as much as
themselves

out of all mentions by row x, how many are from column yCrupi, Mejova, Tizzani, Paolotti, Panisson. Echoes through Time:
Evolution of the Italian COVID-19 Vaccination Debate. ICWSM 2022
124

Measuring echo chambers: topics●●●

extract 20 topics using
NMF
compute how much of
the topics is
attributable to eachside
topical convergenceover time

Crupi, Mejova, Tizzani, Paolotti, Panisson. Echoes through Time:
Evolution of the Italian COVID-19 Vaccination Debate. ICWSM 2022
125

Critique
Kubin, E., & von Sikorski, C. (2021). The role of (social) media in political polarization: a systematic review.
Annals of the International Communication Association, 45(3), 188-206.

Deﬁnition of polarization is vague and inconsistent
Most studies are platform-speciﬁc (Twitter), possibly not generalizable
Causality is unclear (studies suggest both ways, and even none)
No standard measures (related to lack of deﬁnition)

126

Demo
1.

Stance Detection using Network
Partitioning
Topic: Ukraine-Russia conﬂict

2.

Stance Propagation using Network
Clustering
Topic: Vaccination debate
127Discussion
Pay attention to the limitations and strengths of each approach!
Lexicons have higher interpretability but have troubles detecting sarcasm.
In large scale analysis the results are stable.
Are there causal eﬀects in the interplay of morality and political views?
We don’t know! But our values do change over time aaccording to our exposure to events.
The moral content of lemmas may vary according to the context!
‘brotherhood’ has a low impact on betrayal moral the ALM domain
but a considerably higher impact in BLM!
Liscio, E., Araque, O., Gatti, L., Constantinescu, I., Jonker, C., Kalimeri, K. and Murukannaiah, P.K., 2023, July. What does a Text
Classifier Learn about Morality? An Explainable Method for Cross-Domain Comparison of Moral Rhetoric. In Proceedings of the
61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 14113-14132).
128

Discussion
Is detecting the presence of a moral foundation enough?
Should Morals foundations be used as a presence (virtue yes/virtue no), or polar (virtue/vice)scale?
E.g. Fairness/Cheating
Support for fairness and equality/ Refrain from cheating or exploiting others
C: All citizens should have free access to healthcare
F: Only taxpayer should have access to healthcare

129

Discussion
Ok, so we assessed morals in text. Now what?
Important for successful communication campaigns!
Health Organisations create campaigns
addressing their own core values (care)!
However people are concerned about theirFreedom of choice (liberty) & holistic
therapies (purity)!

130

Kyriaki Kalimeri
Giovanni Da San
ISI Foundation
MartinoPh.D. Brain andUniversity of Padova
Cognitive Sciences
Ph.D. Computer@KyriakiKalimeri
Science@giodsm

Yelena Mejova
ISI Foundation
Ph.D. Computer
Science@YelenaMejova

Oscar Araque
Universidad Politécnica
de Madrid
Ph.D.
Telecommunication
Engineering@oaraquei

Webpage to the tutorial

131