"A Tale of Two Movements": Identifying and Comparing Perspectives in \#BlackLivesMatter and \#BlueLivesMatter Movements-related Tweets using Weakly Supervised Graph-based Structured Prediction
Authors: Shamik Roy
Dan Goldwasser
Venue: EMNLP 2023 Findings (2023)
Keywords: characterization of social movements
social media
discourse analysis
perspective identification
TL;DR: Weakly supervised characterization of the discourses on #BlackLivesMatter and #BlueLivesMatter movements using graph-based structured prediction.

Abstract: Social media has become a major driver of social change, by facilitating the formation of online social movements. Automatically understanding the perspectives driving the movement and the voices opposing it, is a challenging task as annotated data is difficult to obtain. We propose a weakly supervised graph-based approach that explicitly models perspectives in \#BackLivesMatter-related tweets. Our proposed approach utilizes a social-linguistic representation of the data. We convert the text to a graph by breaking it into structured elements and connect it with the social network of authors, then structured prediction is done over the elements for identifying perspectives. Our approach uses a small seed set of labeled examples. We experiment with large language models for generating artificial training examples, compare them to manual annotation, and find that it achieves comparable performance. We perform quantitative and qualitative analyses using a human-annotated test set. Our model outperforms multitask baselines by a large margin, successfully characterizing the perspectives supporting and opposing \#BLM.

"A Tale of Two Movements": Identifying and Comparing Perspectives in #BatterivesMatter and #attter Movements-related Tweets using Weakly Supervised Graph-based Structured Prediction
Shamik Roy"
Dan Goldwasser
Purdue University
Purdue University dgo ggol.dwas@@urdue... edu
roy98@purdue. edu
Abstract
Tweet
These thugslhaa chocked the [innocent man until he died. #blacklivesmatter stemolishthepolice
Social media has become major driver of so- cial change, by facilitating the formation of online social movements. Automatically derstanding the perspectives driving the move- ment and the voices opposing a challeng- ing task as annotated data is difficult obtain. We propose a weakly supervised graph-base based approach that explicitly models perspectives in BeaackLvveMaattterreebleee
Tweet
#bluelivesmatter trends only when a [Black person studo is killed. #raciminnamrrica
[They Int Supporter, are destroying our communi #blacklivesmetter terrorist group.
Tweet #1
[Thugs] BLM Suppetter are taking over merica. Hang them Athugs #terroists
Bhat ivedMaller
tweets. Our pro- posed approach utilizes a social-linguistic rep- resentation of the data. We convert the text to a graph by breaking it into structured elements and connect with the social network au- thors, then structured prediction is done over the elements for identifying perspectives. Our approach uses small seed set of labeled ex- amples. We experiment with large language models for generating artificial training exam- ples, compare them to manual annotation, and find that it achieves comparable performance. We perform quantitative and qualitative anal- yses using human-annotated test set. Our model outperforms multitask baselines by large margin, successfully characterizing the perspectives supporting and opposing #BLM.
Figure 1:
Author stances help disambiguate entity refer- ences and understanding perspectives.
of killing of Trayvon Martin, an unarmed Black teenager. While support for the movement fluctu- ated (Parker et al., 2020) and varied across differ- ent demographic groups (Horowitz and Livingston, 2016), since May 2020, following the murder of George Floyd, support has increased across all U.S. states and demographic groups (Kishi and Jones, 2020; Parker et al., 2020), changing public discourse (Dunivin et al., 2022) and resulting widespread protest activities (Putnam et al., 2020). At the same time, these activities have attracted counter movements (Gallagher et al., 2018; Blevins et al., 2019). While #BlackLivesMalie protested against police violence towards Black individuals, the counter movement, referred to using its most prominent hashtag, #BlueLLvesMatter, emphasized the positive role and need for law enforcement, often painting protesters as violating the law.
1 Introduction
Social media platforms have given a powerful voice groups and populations demanding change and have helped spark social justice movements. The platforms provide the means for activists to share their perspectives and form an agenda, often result- ing in real-world actions. Such movements can also lead to the formation of reactionary movements, aiming to counter the call for change. In this paper suggest a computational framework for analyz- ing such instances of public discourse, specifically looking at the interaction between #BlackLivesMat- ter and its reaction #BlueLivesMatter.
To account for the differences in perspectives between the two movements in a nuanced way, we suggest structured representation for analyzing online content, corresponding to multiple tasks, such capturing differences in stances, disam- biguating the entities each side focuses on, as well as their role and sentiment towards them. We dis- cuss our analysis Section 2.2. From a technical perspective, our main challenge is to model the dependency among these tasks. For example, con-
The first movement was formed 2013 in re- sponse to the acquittal of George Zimmerman
*The work was done when the first author was at Purdue University.

---

Authors
31,704
Abstract Entities
Common Perspectives in
# Retweet relations
3,206
Pro BlackLM
Pro BlueLM
4 Keywords in author profiles
9,500
# Tweets
402,647
Black Americans
Positive Target
Negative Actor
Tniity mentioned in tweets Hashtags used in tweets
393,441
Positive Actor,
Police
Negative Actor
1,068,525
Positive Target
Time span
05/26/20 to 06/26/20
Community
Target
Racism
Negative Actor
N/A
Table Unlabeled #BLM corpus statisties,
Democrats
N/A
Negative Actor
sider the use of the entity "thugs" in Figure 1 by two different authors. Understanding the perspectives underlying the texts' stance towards #BlackLives- Matter, requires disambiguating it as either refering police officers, to ers. We model such interaction among tasks using graph-basee based representation, in which authors, their posts and its analysis are represented as nodes. We use a graph neural network (Schlichtkrull et al., 2018) to model the interaction among these ele- ments, by creating graph-contcxtuullized node rep- resentations, which alleviate the difficulty of text analysis. We discuss our graph representation in Section 3.1 and its embedding in Section 3.2.
Republicaus
N/A
Positive Actor
Government
Nogative Actor
N/A
White Americans
Negative Actor
NAA
Positive Actor.
BLM Movement
Negative Actor
Positive Target
support-
Petition
Posirive Target
Antifa
Negative Actor
Table 2: Common abstract entities and perspectives in BlackLM and BlueLM campaigns. Only 29% entities in the corpus are covered using an exact lexicon match approach for entity disambiguation.
on the BLM protest and the counter- protests. This dataset was collected using keyword matching and spans from 2013 to 2021. However, as shown in the paper, BLM-related tweets spiked mainly after the murder of George Floyd in 2020. Hence, in this paper, we study the tweets that were posted in month time span following George Floyd's murder. We consider original tweets written in English and discard any author from the dataset who tweeted less than 5 times in the timeframe. The dataset statistics can be found in Table 1.
counter-protests.
Social movements are dynamic, human-annotated data for training costly. Hence, we explore learning from weak supervision, ini- tialized with artificially generated data using Large Language Models (LLMs) (Brown et al., 2020) and amplified using selfntraining, exploiting the graph structure to impose output consistency. We discuss self- in Section 3.5.
result, using
To evaluate our model and the quality of artificial data, we human annotate ~ 3k tweets by both sides for all analysis tasks. We compare our graph-based approach with multitask and discrete baselines in two settings direct supervision (train with human- annotated data) and weak supervision (train with artificial data). Our results on human-annotated test set consistently demonstrate the importance of social representations for this task and the potential of artificially crafted data. We also present an ag- gregate analysis of the main perspectives on 400k tweets related to the George Floyd protests.
2.2 Defining Perspectives
Capturing Perspectives. Previous study (Giorgi et al., 2022) has focused on topic indicator key- words for understanding differences in perspectives among BLM and counter-movements However topic indicator lexicons might not capture the in- tended meaning. For example, the intent of hashtag use is often ambiguous, shown in Figure 1, both authors use "#blacllivsmmateer to express oppo- site perspectives. Previous studies (Rashkin et al., 2016; Roy et al., 2021; Pacheco et al., 2022) have shown that sentiment toward entities can disam- biguate stances in polarized topics, we follow this approach and adapt the Morality Frames proposed by Roy et al., 2021 that are linguistic frames that capture entity-centric moral foundations. Adapting this work, we use four dimensions of perspectives towards a specific entity actor ("do-er", having agency) and target ("do-ee", impacted by the ac- tions of the actor), and sentiment, positive or neg- ative, based on the author's 's view (e.g., negative
2 BLM Data Collection and Analysis
First, we describe the data collection process for the #BLM movement. Then, we define the structured perspective representation used for analyzing it.
2.1 Dataset
For studying perspectives on the BLM movement we use the dataset collected by Giorgi et al., 2022. This dataset contains tweets in various languages

---

Predicted Labels
3.1 Structured Representation of Text
Tweet Stance
Entity Role
Sent. to Emiity
Entity Mapping
To jointly learn perspectives and identify stances on #BLM, define perspectives how entities portrayed by author as described in Section 2. To identify perspectives towards an entity in a tweet we need to identify - (1) the abstract en- tity the targeted entity maps to, and (2) the role assigned and sentiment expressed towards the en- tity. Our insight is that the entity mapping, entity role/sentiment, and author stance all are interdepen- dent decisions. For example, if an entity maps to the abstract entity "police" and the author's stance is "pro-BlackLMM" then the sentiment towards the entity will most likely be "negative" and the en- tity role will be "actor". Similarly, if an author is identified to be repeatedly mentioning "police" as a negative actor, then an unknown entity men- tioned by the same author, that is identified to have role of a "negative actor" will most likely address "police". To explicitly model these dependencies between authors' stances and perspectives, we first convert the large unlabeled tweet corpus (in Ta- ble 1) to a graph consisting of textual elements (text, entity mentions, hashtags). Then we con- nect the textual graph to the social context (author retweet graph). A unit representation of our multi- relational graph is shown in Figure 2 and various nodes and relations among them in this graph are described below.
Layer3
Layer
FC Layor
Concatenation
Concatenation
Rule Based Consistency Check
Author Embed.
Tweet Embodding
Entity Embedding
Clssiiters trained on 000
Classifir
Sentiment
author 2
retweeted
author 1
Twereted
tweet
Useed profes
Classffeer
keyword
hashtag
Multi-Rtelaional Unit Graph
entity
Unlabelee
Self-Infurred Consistent Train
Few Training
ABLM Corpus
(coffted/annotated]
Figure Proposed self-supervised model.
actor, will have a negative impact on the target). We characterize the moral reasoning behind that sentiment using Moral Foundation Theory (Haidt and Joseph, 2004; Haidt and Graham, 2007), how- ever, given the difficulty of learning abstract moral dimensions without direct supervision, we rely on external tool and report the results an aggregate.
Entity Disambiguation. The focus on specific entities is central to the movements' messaging. However, identifying it directly from text can be challenging. For example, in Figure 1, 'thugs' map to both 'police' or 'BLM Supporters' depending on the speaker. To alleviate this difficulty, we identify the key abstract entities discussed by both sides using human-in-the-lopp data analysis an aggre- gate level (listed in Table 2). The description of abstract entities and the procedure for identifying them can be found in Appendix A. We define an entity disambiguation task, mapping the raw enti- ties that appear in tweets to the intended abstract entity.
(1) Author-tweets- Tweet: An author node con- nected to the tweet nodes they tweeted.
(2) Luthor-retweets-Author Author nodes can be connected to each other with retweet relationship. (3) Keathoor-uss-KKyworrr An author node can be connected to a keyword node that mentioned their profile description. Keywords are meaningful hashtags and hashtag-like phrases. For example, "Black Lives Matter" is a phrase, and "#blacklives- matter" is a similar hashtag.
A hashtag node is connected to the tweet node is used in.
(4) Besshaa-ussd-in-T11
Stance. We study the perspectives from two oppos- ing standpoints- Pro- Pro-BlackLivesMattee (addressed (ad-
(5) intitt-mentioned--i--Wea connected to the tweet node it is mentioned in.
An entity node is
M)) and dressed as pro-BlueLM).
3.2 Learning Representation using GCN
3 Weakly Supervised Identification of Perspectives
After converting text to graph and connecting it with the social network of authors, we learn the representation of the graph elements using a multi- relational Graph Convolutional Network (R-GCN) (Schlichtkrull et al., 2018), an adaptation of tradi- tional GCN (Kipf and Welling), where activations
In this section, we propose self-supervised mod- cling approach for a holistic identification of per- spectives and stances on MavectLivesMatter.

---

from neighboring nodes are gathered and trans- formed for each relation type separately. The rep- resentation from all relation types is gathered in a normalized sum and passed through an activation function (ReL.U to get the final representation of the node. A 2-layer R-GCN is sufficient in our case to capture all dependencies, resulting in rich- composite representation for each node.
3.4 Entity Role and Sentiment Priors
The "mentioned-in in" relationship between an entity and the tweet it is mentioned in, may have multi- ple types e.g., pos-actor, neg-actor, and pos-target, based on the sentiment expressed and the role signed to this entity. We initialize the priors of this relationship type in our graph using two off-the- shelf classifiers, Csent and Crotes for sentiment and role classification, respectively, trained on out-of- domain data (OOD). They are defined as follows.
3.3 Predictions on Learned Representation
We define a set of prediction tasks on the learned representation of the nodes using R-GCN. These prediction tasks help infer various labels for the graph elements and at the same time maintain con- sistency among various types of decisions. In the following objectives, E' represents learnable rep- resentation of a node using R-GCN, represents trainable fully connected layers, a represents Soft- max activation, and ( means concatenation.
seent/rate (E2) a (HH (E2))
We align predictions from Csents Crate (on the large unlabeled corpus) the "mentioned-in" edges using the following loss functions.
Lsa m (a EE @ Ee),, Ceent (E
Lra m (o @ E,)), Crote (EO))
Tweet stance prediction: Intuitively, the stance in a tweet depends on its author. Hence, for tweet, 4, and its author, a, we define the classification task of tweet stance (pro- BlackI BM-eelM))M follows which ensures consistency among author stances and their generated tweet stances.
Here, E is the non-trainable input representation of an entity and represents L1 loss. These learning objectives set up the priors for entity sen- timent and role in our framework. training, these priors get updated as the parameters of the classifiers Csent Crote are also updated.
course of
slueed-stance o (HHuweel-stance Ea + E()
We define the final loss function J. as the sum of the alignment losses, Lea and Lra and all predic- tion losses described above, generally denoted P(g. y) where ğ is the predicted label, y the gold label, and P is CE loss. We optimize L by updating all trainable parameters including R-GCN.
Entity sentiment, role, and abstract entity la- bel prediction: The sentiment and role of given entity depend on the tweet text it is mentioned in. Given entity, C, and the corresponding tweet, I, we define the classification task of the entity senti- ment (nos./neg.) and role (actor/target) as follows.
3.5
Self Learning Loop
Nsent o HHsent Ee + Ex))
We propose self-learning procedure where our model checks consistency among all predictions after every learning steps which we name the "Inference step". The model adds elements (e.g., tweet, entity) with consistent labels found in this step, to its training set for the next iterations. In this manner, our model able to start with only a few annotated examples as a training set and keep annotating data and increasing training examples in a continuous learning loop. We permit k epochs after each inference step. 1j. so that the model parameters are learned based on the new training examples inferred at the previous inference step, 1y-1- The model identifies consistent labels using the following consistency checks sequentially. (1) Label confidence: A label reliable if pre- dieted with a minimum confidence of [o, 1]. Only reliable labels are used the next checks.
Drote oHHloate EE + Ex))
Intuitively, the abstract entity label does not directly depend on the tweet text rather depends mostly on its textual properties and indirectly on the sen- timent/role assigned to it. We define the abstract entity label (out of I entities in Table 2) prediction by conditioning only on the entity representation.
Dmap a (Hmap(Ee))
Additionally, to maintain consistency between stance and perspectives towards entities (e.g. pro- BlackLM and "police neg. actor" consistent), we define a prediction task of stance on the learned representation of entities and tweets as follows.
Trentity stance = o (eetitty staner E, ① E2))

---

<pronpt-start>
that 1. Explicitly nention one or more entities fron the following list [Black nan, Black people, Black uonan, Black community] and portray then positive targets.
Write different
2. Be the support the "Black Lives Matter" noverent.
3. Use one nore hashtags from the following list: [#blacklivesmstter, #bin, #policoorutallity, #racisn].
Answer:
spronpt-end>
sgeneration-stane
1. stand with the black comunity and the "Black Lives Natter" novement. #blecklivesnatter Hbln
We continue fight for justice and equality for all black people. halllivesnatter aboliceerutalllyaaddeei
3. No one should have live fear because
color of their skin. #olackllvesnatter #racise
4. All black Lives natter from the oldest to the youngest. We nust protect then all. #b1n #pclicebrutality
nust support 6. We nust work together to end the systeratic racisn and oppression of black people. #blacklivesnatte drrccim black people those in positions power. #oin
wonen
backbone
comuunity
elways. #blacklivesnette
longer tolerate the unjust
8. It's tine for change нe need to see more black people in leadership positions. #elacklivesmatter
The fight for racial equality is not over until all black lives netter, Welllivesnatter upolicebrutality
nust together to stand up against racise and projudice
farms. #bllackiveesatteer
Figure 3: Prompt example for GPT-3 to generate tweets having "pro-BlackLMM" stance and portraying "Black Americans" as "positive target". The black-colored text is the imput text in the prompt and the blue-colored texts are generated tweets by GPT-3. The generic prompt structure can be observed in Figure in Appendix B.
(2) Tweet consistency: A tweet is consistent the predicted stance of the tweet is consistent with the perspectives identified towards all mentioned entities in the tweet (determined from Table 2). For example, tweet stance pro- BlackLM is consistent with mentioned entity in the tweet identified as "police-neg-acteee and inconsistent with an entity predicted as
Artificial Training Data Generation: For gener- ating tweets containing specific perspectives, we prompt LLMs (Brown et al., 2020) in way that all of the structured elements for the perspectives present in the generated tweets. For example, to generate tweets that "pro-BlackLM" and por- tray "Black Americans" as "positive target", we prompt LLMs to generate N tweets that meet 3 conditions (1) explicitly mention one or more en- tities from ('Black man', 'Black people", 'Black woman' etc.) and portray them as positive tar- gets, (2) are in the support of the "Black Lives Matter" movement, (3) use from (#blacklivesmatter, #blm, etc.). We follow this prompt structure for each (stance, abstract en- tity, perspective) tuple in Table 2. We find that the LLM-gencrated tweets are pretty realistic. An example prompt and some generated examples are shown in Figure 3. The generic prompt structure can be observed in Figure 7 in Appendix B. We convert the generated artificial training examples to graph structures before. For each (stance, abstract entity, perspective) pair construct imaginary author node in the graph whose embed- dings are initialized by averaging the correspond- ing artificial tweets. this manner, we get the same unit structure for the artificial tweets as the real tweets. We experiment with two LLMs. GPT- 3 (Brown et al., 2020) and GPT-J-6B (Wang and Komatsuzaki, 2021). We observed that GPT-J gen- erated mostly repetitive examples, hence, the GPT-3 generations. Detailed prompting meth- ods, generated data statisties and examples, L.LM
(3) Author consistency: An author is consistent if all of their consistent tweets (identified in (2)) are labeled to have the same stance Apro-BlackL.M or pro-BlucLM) and they have at least such tweets. Tweets from the consistent authors and the corre- sponding consistent entities are added to the train- ing set for the next learning steps. We also keep training the off-the-shelf sentiment and role label classifiers (Csent, Crote) each step, using the in- ferred training set by our model so that they are up-to-date. We keep running our model until the model does not generate more than y% new train- ing data for consecutive æ inference steps.
more hashtags
4 Training and Test Data Collection
Detecting perspectives in social media texts us- ing supervised models requires costly human- annotated training data. Hence, being inspired by the recent advances in generative Large Language Models (LLMs) (Min al., 2021), we artificially craft training examples using LLMs. To evaluate the quality of the crafted training data and to evalu- ate our model's performance in real test data, we human annotate a subset of real data.

---

hyperparameters, and pre/post-processinggg steps can be found in Appendix B.
5 Experimental Evaluation
5.1 Experimental Settings
Human Annotation of Real Data: To evaluate the quality of the artificially crafted data and to eval- uate our model's performance real data, we hu- man annotate subset of the real data. We sample 200 authors from the corpus described in Table I and human-annotate these authors and their tweets for stances (pro-BlackL waa--BllckkM/// Ma/pooo-/ooooBLLAAA
We first perform task adaptive pretraining (Guru- rangan al., 2020) of RoBERTa (Liu al., 2019) using unused #BLM tweets from the dataset by Giorgi al., 2022 (addressed as RoBERTa-tapt), We use it for implementing the baselines and ini- tializing the nodes in our model. We observe im- provement in tasks using RoBERTa-tapl over basic RoBERTa (in Table 4). We use RoBERTa-baad classifiers as external classifiers, Csentt and Croie- and pretrain them out-of-domain data proposed by Roy et al., 2021. Details of our model initializa- tion, hyperparameters, stopping criteria, pretrain- ing Csent and Crotes etc. are in Appendix D.3.
We also human--nnotaa the entities they mention in their tweets for sentiment towards them (pos./neg.), their role (actor/target), and their mapping to the abstract entities (one of the 11 abstract entities in Table 2). Each data point is annotated by two human an- notators and we find substantial to almost perfect agreement in all the annotation tasks. We resolve disagreements between the two annotators in all of the annotation steps by discussion. We observe that often the supporters of the movements "hijack" (Gallagher et al., 2018) the opponent's 's hashtags to troll or criticize them. We annotate these tweets as "Ambiguous Tweets" and are identified by look- ing at keyword usage (e.g., when a pro- blackLM tweet uses the keyword "bluelivesmatter"). De- tailed human annotation process, inter-annotator agreement scores, examples of ambiguous tweets, and per-class data statistics are in Appendix C.
Baselines: Our task of the joint identification of perspectives in terms of sentiment toward main ac- tors, their roles, their disambiguation, and stance identification, is unique and to the best of our knowledge, other works studied unified per- spective detection in such a setting. Hence, the closest baseline of our model is the multitask mod- eling approach (Collobert and Weston, 2008). We compare our model with a RoBERTa- based multi- task approach where a shared RoBERTa encoder is used and fine-tuned for the identification of entity sentiment, role and mapping, and tweet stances. To compare it with our proposed model that in- corporates author social-network interaction, we enhance the multitask RoBERTa by concatenat- ing vodial-network-eehhnneee author embeddings with text representations. We also compare with discrete RoBERTa-baue text classifiers where all tasks are learned separately by fine-tuning a sep- arate RoBERTa encoder for each task. We also compare with the wwwworddmatching-baseee base- line for stance classification (Giorgi al., 2022). For fair comparison, and role classification baselines on the same out- of-domain data by Roy et al., 2021 that we use for pertaining Ceent and Crole- Details of the baselines and their hyperparameters are in Appendix D.4.
Authors
of Twwels
Ambignous Twerts
Entilies
L.AM (GPT-3) Generaled
582
517
TRAIN (Work. Sepervision
Human Annolated (Direct Supervision)
Human Annotated
129
Table 3:
Training and test data statistics.
Train-Test Split: We randomly sample small sub- set of the human-annotated authors and use these authors, their tweets, and mentioned entities as training set and the rest of the data a test set for our proposed model and all baselines. We per- form our experiments two setups (1) Weak su- pervision: L.M-ggenrrted examples are used for training, (2) Direct supervision: human-annotated real data are used for training. In both setups, the models are tested on the human-annotatee test set. The statisties for the LLM-generated train set and human-annotates train/test set are shown in Table 3. Note that our self-learning-basee based model depends only on few training examples for initial supervi- sion, hence, few training data in both settings are enough to bootstrap our model.
pretrain all sentiment
5.2 Results
We present all individual classification results in Ta- ble 4. Our first observation is that the identification of stances in the ambiguous tweets is difficult as, by definition, they "hijack" (Gallagher et al., 2018) the opponent's keywords. As a result, the keyword- based baseline performs best among all baselines in overall tweet stance classification, however, it

---

MOneEX
Weok Sep.
Direct
Weak
Dist
Weak
Dirtt
Weak Sup.
Diveet
Week
Direet
Wook
Divect
NAIV
Kunsee
Koyood Read
55.93
ST.77
15-12 RLS
51.T
SU 10
20.00 1.4
1:31..15
77.55-11.6
10.00
00 4-50
51.3
80
94
56.35 0.5
17.04-35
15.71 L.3
3.5
07.08 0.5
76.30-1 2.
50.33 1.
05
637
15:1-15
67
MELTITASK
79.07 2:30
00.28
1.5
78:01:44 6.7
06.30-2:1
55 1.2
22122-11
92:22:32
44.31231
Astur Einbed
01.11 1.1/1
10.15
1200
70/03 2.3
00.06
31.58
07
0071111
1310.108
17.17..0
09:0
1:22
OUR MODEL
AArber Netwass
Gooph
2:2
10.29 1.1
+.0
75.05
89:72
10.06:100
36.11 01
2632202
5468 1:03
86.8
5001 03
25.29
62.74120
SSrt awiing
00.83-1.3
90.36-1.8
05.07-0.5
6G.80-28
63.45-555
0047-46
57.000.5
67.03-04
50.73-0.6
87.09-0.3
51.18-344
(8:00-33
Table 4: Average macro F1 scores for classification tasks seeds (weighted FI scores are shown in Appendix D.5). Entity mapping is an 1 11-class and the rest are 2-class prediction tasks. Author stances are determined by majority voting of the predicted stances of their tweets. Amb. means Ambiguous.
test set over rums using 5 random
WEAR SUPSEVISION
DIRECT SUPrRvISION
help in ambiguous tweet stance detection, as in this setup, the authors are not real but rather imitated and their embeddings are just the average of the generated tweets. Also, ambiguous tweet examples are not present in the LLM-generated train set. We present the combined perspective identifica- tion results in Table 5 and observe that our model outperforms the multitask baseline in almost all perspectives. We observe low Fl scores for a few perspectives such as "Black Americans neg actor", "BLM pos target", and "Republicans pos actor". In the direct supervision setup, we find that "Black Americans neg actor" (Precision: 60%, Re- call: 2.7%) and "Republicans pos actor" (Precision: 58%, Recall: 12%) get overall low average recalls. We conjecture being a less popular perspective (as shown in the support column) the reason for low recalls. We find that 48% of the time "Republicans" wrongly map to the abstract entity "Government". In the timeline of this dataset, a Republican presi- dent was in power in the US. Hence, the framing of Republicans was probably as government figures. In the case of "BLM", 70% of the time it is mapped to related abstract entity, "Black Americans".
PERSPECTIVES
Model
Mullitsk
Our Medel
Multitkk
Sure.
reg ootrr
25%
Whlte Artricass neg acur
02.70:1.6
10.60 9.5
Black Americats pos lugst
72.00
75.09:2.1
$4.0
0.
443
Racism ngg aoor
06.80.33:0
51.02 8 5.9
10.07:4.1
00.55 1.7
130
BLM pi adur
49.45 45.45:24.5
25. 16
5.5
19.52-1.3
6.
64
Covemment neg scier
37.58 8.5
15.70
5.7
02.76 7.0
1101
Democnas neg acorr
62.80
54,06
2: 3.4
07.58:3.5
5.
90
BLM pi lagel
20:1 0.5
25.04:6.1
10.30:5.9
0,0:000.
29
Communttio tagget
42.87:3322
12.13
2.5
54.11 0.0
07:36 4.2
Police
46.81:22.
45.9
1.4
Police po
64.171177
01:08:1.9
3..34:3..1
pis tarpe
Republlcars pos actor
21:1::6.4
29.85:55.5
40.0 1..
BLM ncg aclor
28.09.24.2
15:00
2..
Amtisa rog sdur
45:8 1/7
46.54:1.00
61.88-3.8
63:31-22.11
Black Americas nog actor
26.61.18.00
19:00 3: 3.4
5.16 3.2
1.00 21
37
Avo. WEIGHTEO F1
00.27/12.3
51.13
1.8
65.88-144
3.7.0.50
1
Table 5: Fl scores for perspective identification by our model and Muiiissk-RBBBRRT---ptAAuuoooooo
fails in ambiguous tweets. Next, we observe that performance improves in all tasks when all inter- dependent tasks are learned jointly in the multitask baseline compared to discrete classifiers. In mul- titask setup, adding author embeddings improves performance bit in some tasks.
We study our model's performance in four set- tings (more details in Appendix D.5). We observe that by just converting text to graph we get an im- provement over discrete text classifiers over initial text embeddings (from RoBERTa-tapt), It proves the advantage of studying texts as graphs consist- ing of structured elements and performing joint structured prediction on that. Next, we obtain a large gain in tweet stance classification perfor- mance when the author network is added to the text-only graph. Finally, adding the self-learning loop improves the performance a lot in all tasks and outperforms all basclines proving the effectiveness of our approach.
Finally, we evaluate how much GPT-3 generated or
data is required for effective training of the models and find that our model is less sensitive to the number of training examples in both cases compared to the multitask baselines. The learning curves can be observed in Figure 4. The details of the ablation can be found in Ap- pendix D.8.
5.3 Qualitative Evaluations
We infer perspectives and stances in the whole cor- pus (in Table 1) using our model and the multitask baseline and perform the following qualitative eval- uations (more evaluation details can be found in Appendix E).
The trends are mostly the same in the weak su- pervision setup and it achieves overall compara- ble performance with the direct supervision setup. In this setup, adding author information does not

---

70
Literal Entities
Most assigned perspectives by our model (direct sap.) In pro-#Rlued M
0 65
In pro-FRlackI.M
31 58
60
Blakk Victins
blacks pos. target
blacks neg. sctor
Derek Chaavin
police neg.
NiA
Thugs
police neg. actor
antifa neg. astor
/ 35
David Dom
blocks pos. target
polise pos, target
25
Lives
blacks pos. langei
comm. pos. target, police pos. turget
35%
67%
100%
16 Training Data
Dunald Trump
government actur
republicons pos. sotor
-M- Multitask (Wealt Sup.)
Multiaak (Direct Sup.]
They
blacks pos. target
dems. nog. actor, antifa neg, actor
Model (Wesk Sup.)
Model (Direct
(a) Perspective detection
Table 6: Examples of literal entity abstract entity map. Sometimes pro-Blackl . and Ml--BlueLMM use the same phrase to address different entities and/or perspectives.
Aug. Maxro #1Score
95
90
85
DISCOUBSE iS Pno./Bl.catavvssa Gedosed ssisg ou model and diac sap.)
Prespections
Prrspectives
Eeamplee
Contect
Cantest
70
33%
100%
pulssagg babe.
pos. tepel.
cups the murdered
#Brocand
Training
bln
beya.
pos. urget.
- Multitosk (Weak Sup.)
- Multitok (Direct Sup.]
big America
realy
Our Model (Mteak Sup-)
Model (Direct
DISCOURSE PRo-PBLsLLovssAAA (infened using our model and direct spp)
(b) Tweet stance detection
police paw. sakor
tepatshiny.
rids neg actre
polis pas, uaps, an-
Protect the ullisns They are only fol-
lowing underseed lorping Ameria
Figure 4: Learning curves for perspective and tweet stance detection for our model and the Multitask base- line.
Dag uadie
novenest Aubs.
byyahhetrrry
desuorraaa nay. untie
antifa neg.
actor.
They was foandel by Democntss
Baa works to divisk people.. tut units.
FBLM
muist reganization
Table 7: Discourse of movements explained with mes- saging choices and Moral Foundations (MFs). Moral Foundation care/harm was used in all of the cases by both sides. Hence, it is removed from the table.
Point Skeerial Corr. Coeft
0.55
0.35
Carl (Pro SlacciM Follow Bevne)
Corr (Pre- SlaesMt kollow Aepe)
Carr (Pro slackuMt Stare Left Medial
Corr (Phe BlastA Share light Media)
address different entities as shown in Table 6. Pro- BlackLM addresses police as "thugs" while pro- BlueLM addresses Antifa as "thugs". When pro- BlackLM tweets mention "lives", they mean Black lives while the Pro-BlueI.M means the lives of po- lice officers. These patterns are better captured by our model compared to baselines (comparison and example tweets are shown in Appendix E.2).
IB KeywarddBBssed
Multitask (week sup.) Our Model (direct sup.)
Muultttsk (direct sup.)
B Our Model /weak sup.)
Figure 5: Correlation hetween stances on #BI.M move- ment and authors' following and sharing behaviors.
Correlation of author behavior with stance: We examine the correlation between authors' following and sharing behavior on Twitter with their stances on #BLM movement. We find that there are posi- tive correlations between being Mro-BlacLLM and following Democrats and being pro-BlueLM and following Republicans on Twitter. We find similar correlations in sharing behavior of the authors from left and right-biased media outlets and supporting BlackLM and BlueLM, respectively. These corre- lations are consistent with these political camps' stances on #BLM. However, this information not used in the learning phase, hence, it serves as an extrinsic evaluation for our model on the whole corpus. We observe that these correlations increase using our model's 's labels compared to baselines as shown in Figure 5, validating our models' predic- tions.
Discourse of the movements: In Table 7, we sum- marize some high PMI perspectives identified in each of the movements, the corresponding moral foundations used in context of these perspectives. and the other perspectives that frequently appear in the same tweet (full list is in Appendix E.3). We infer moral foundation labels in tweets using an off- the-shelf classifier. We find that the perspectives can explain the discourse of the movements. For ex- ample, in pro-BlackL.M when police are portrayed as negative actors, Black Americans are portrayed as positive targets, and BLM movement as a posi- tive actor. Moral foundations fairness/cheating and authority/subbeesion are used in this context. In contrast, in pro-BlueLM, police are portrayed as positive actors and targets, and in the same context, Antifa is portrayed as a negative actor. The moral foundation of loyalty/betrayal is used in context.
Entity mapping analysis: Depending on the au- thor's stance, the same phrases are often used to

---

NLP. Early works studied the problem of identify- ing the names of civilians in news articles killed by police using EM-based (Keith et al., 2017) and Deep Learning approaches (Nguyen and Nguyen, 2018). Recently, Ziems and Yang, 2021 introduced news corpus covering 7k police killings to study entity-centric framing of victims, defined as the de- mographics and other status (e.g.. armed/unarmed), A shared task was proposed for identifying BLM- centric events from large unstructured data sources (Giorgi al., 2021b), and Giorgi et al., 2022 in- troduced a large #BLM tweet corpus paving the way for more studies in this area. In this paper, we propose a holistic learning framework for under- standing such social movements.
of
6rit/50 e/anno wriiao
% Pro- BlueLM
0% % Pro- BlackLM
(a) Tweeting trend in each camp.
Usage
5/102000
61/2000
enusna
wns/ana
e/19/000
blm-neg-aue actor
Bems-neg-acee actor
(b) Trend in portrayal of entities as neg. actors in pro- BlueLMM
0.25
Usage
0.55
Our work is broadly related to stance detec- tion (Küçüü and Can, 2020; ALDayel and Magdy, 2021), entity-centric sentiment analysis (Deng and Wiebe, 2015; Field and Tsvetkov. 2019; Roy al., 2021), entity disambiguation (Cucerzan, 2007; Ganea and Hofmann, 2017; Eshel et al., 2017), data augmentation (Feng et al., 2021), and the works that analyze similar discourses on social media (Demszky et al., 2019) and incorporate social super- vision in language understanding such as sentiment analysis (Yang and Eisenstein, 2017), political per- spective detection (Li and Goldwasser, 2019), fake- news detection (Nguyen et al., 2020; Mehta et al., 2022), and political discourse analysis (Pujari and Goldwasser, 2021; Feng et al., 2022). Detailed discussions on the CSS studies on the movements, stance, perspective analysis and data augmentation techniques can be found in Appendix F.
0.s
sparaaz
5na/2020
6/3/2000
67/2022
0711/2020
5/13/300
623/2000
6729/2000
eatram0
67322020
tasttee-
community tartttt
(c) Trend in portrayal of entities as pos targets in pro- Bluel.M..
Figure 6: Temporal trends identified using our model and direct supervision.
However, high-level moral foundations in both of the camps are sometimes similar (e.g., care/harm is frequently used with all perspectives) and entity perspectives resolve the ambiguity in those cases. Temporal trend: Using our model (direct super- vision), we find that 20% of the tweets are identi- fied as Pro-BlueL.M and the rest as pro-BlackL.M, We find that first responders following George Floyd's killing were the pro- Pro-BlacKLM camp. In contrast, the percentage of Pro-BlueLM tweets per day slowly increased over time (Figure 6a). When the protest started (on May 26) following George Floyd's death, the Pro-BlueLMM camp initially por- trayed BLM and Antifa as negative actors (Figure 6b) and communities to be positive targets or suf- ferers of the BLM movement (Figure 6c). As the movement progressed (after June 2) Democrats were portrayed negative actors more equally, Additional trends can be found in Appendix E.4.
7 Conclusion
In this paper, we propose a weakly-supervist self- learned graph-based structured prediction approach for characterizing the perspectives and discourses on the LBlackl.ivesMatte and the (Bluel_ivesMar ter movements on social media. We evaluate our model's performance in a human-annotated test set and find significant improvement over all base- lines. Finally, using our model we successfully an- alyze and compare perspectives expressed in both of the movements.
6 Related Works
Limitations
The discourse of the #BLM movement is mostly studied in computational social science (CSS) us- ing keyword-basee analyses (De Choudhury et al., 2016; Gallagher et al., 2018; Blevins et al., 2019; Giorgi et al., 2022). However, it is understudied in
For the artificially crafted training data generation, we mostly use GPT-3 which is not open source and is available paid service Although our model
hhttps://beta.oo openaii cotpaaaiiccmmppaaruunnd

---

depends only on a few GPT-3 generated texts (cost below $3 USD combined), generating examples at a very large scale will be expensive using GPT-3, Experimenting with the increasing number of open- source LLMs is costly in a different way as they require advanced computing resources to mount and run. Hence, we leave the study on the differ- ence in training data generated by various LLMs future work.
biased texts that contain specific perspectives and stances. Hence, the concerns regarding different types of biases (e.g. racial, national, gender, and so on) are not applicable in the case of our study. Be- cause we prompt LLMs to generate only a few texts that have specific structured properties (c.g., stance, sentiment towards entities), and as described in Ap- pendix B, one author of this paper manually went through the generated examples to detect inconsis- tencies or any unexpected biases in the generation and no unexpected bias was observed. We believe prompting LLMs using the structured way that we propose in this paper is effective in avoiding any in- consistencies that may be additionally incorporated by the pre-trained models.
Our main focus in this paper is to develop a holistic framework that can be applied to different events related to social movements for character- izing perspectives. As result, in this paper, we focus on one significant event related to the #BLM movement which is the outrage right after George Floyd's s killing. However, our model can be applied in the case of other similar social movements that were viral on social media such as the #Me Too movement. We study two opposing movements named BalackLiivesatter and #Bluel.ives in this paper. Extending this study to another rele- vant and potentially more ambiguous slogan, #All- LivesMatter can be interesting.
Human Annotation: We did human annotation of data using in-house annotators (aged over 21) and the annotators' were notified that the texts may contain sensitive phrases. The detailed annotation process and inter-annotator agreement scores are discussed in Appendix C.
In this paper, we carefully ad- dressed all communities of people and movements that appear in the dataset. We made sure that ev- ery entity is addressed with due respect. All of the sentiments, perspectives, and trends reported in this paper are outcomes of the models we devel- oped and implemented and in no way represent the authors' or the funding agencies" opinions on this issue.
Bias and Ethics:
Our model depends on pre-identified abstract entities and perspectives as priors. This pre- identification of abstract entities and perspectives is a semi- aunim-automated process with soman-in-the-looy or requires world knowledge. Making this step fully automatic or extracting them from an existing database can be interesting future work.
Our model does not automatically identify new abstract entitics. However, in real life, new abstract entities may appear over time. This limitation does not affect our study in this paper because the study is done in short time frame of one month and emerging of new abstract entities change in au- thors' stances in this short time frame is unlikely. Extending the model to identify new abstract en- tities in a temporal fashion is our intended future work.
Datasets Used: All of the datasets used in this paper are publicly available for research and we cited them adequately.
Acknowledgements
We gratefully acknowledge Nishanth Sridhar Nakshatri for helping with human annotation. We are thankful to Nikhil Mehta and Rajkumar Pujari for their feedback in the writing. We also thank anonymous reviewers for their insightful comments that helped improve the paper lot. The project was partially funded by NSF CAREER award IIS- 2048001.
Ethics Statement
this section, we clarify the ethical concerns in the following aspects.
GPT-3 Generations: There have been concerns about inherent bias in pretrained Large Language Models (LLMs) in recent works (Brown et al., 2020; Blodgett et al., 2020). As LLMs are pre- trained on large corpus of numan-generated data, they may contain human bias in them. We want to clarify that, in this paper, we use LLMs to generate
References
Abeer ALDayel and Walid Magdy. 2021. Stance detec- tion on social media: State of the art and trends. In- formation Processing Management, 58(4):10259)
Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2020. Do not have

---

enough data? deep learning to the rescue! In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 34, pages 7383-7390.
Processing (Volume 2: Short Papers), pages 438- 444, Beijing, China. Association for Computational Linguisties.
Isabelle Augenstein, Tim Rocktäschel, Andreas Vla- chos, and Kalina Bontcheva. 2016. Stance detection with bidirectional conditional encoding. In Proceed- ings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876-885.
Andreu Casas and Nora Webb Williams. 2019. Images that matter: Online protests and the mobilizing role of pictures. Political Research Quarterly, 72(2):360- 375.
Dennis Chong and James N Druckman. 2007. Framing theory. Anmu. Rev. Polit. Sci., 10:103-126.
Eric Baumer, Elisha Elovic, Ying Qin, Francesca Pol- letta, and Geri Gay. 2015, Testing and comparing computational approaches for identifying the lan- guage of framing in political news. In Proceedings of the 2015 Conference of the North American Chap- ter of the Association for Computational Linguisties: Human Language Technologies, pages 1472-1482, Denver, Colorado. Association for Computational Linguisties.
Kenneth Church and Patrick Hanks. 1990. Word associ- ation norms, mutual information, and lexicography. Computational linguistics, 16(1):22-29.
Alessandra Teresa Cignarella, Mirko Lai, Cristina Bosco, Viviana Patti, Rosso Paolo, et al. 2020. Sardis- tance evalita2020: Overview of the task on stance detection in italian tweets. In Proceedings of the Seventh Evaluation Cumpuign of Natural Language Processing and Speech Tools for Italian. Final Work- shop (EVALITA 2020), pages 1-10. Ceur.
Jeffrey Layne Blevins, James Jaehoon Lee, Erin E Mc- Cabe, and Ezra Edgerton. 2019. Tweeting for social justice in# ferguson: Affective discourse in twitter hashtags. new media & society, 1677):63661653.
Ronan Collobert and Jason Weston, 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceed- ings of the 25th international conference on Machine learning, pages 160-167.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics. pages 5454- 5476, Online. Association for Computational Lin- guistics.
Silviu Cucerzan. 2007. Large-scale named entity disam- biguation based on wikipedia data. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP- (EMNLP.CONLL), pages 708-716.
Amber Boydstun, Dallas Card, Justin H. Gross, Philip Resnik, and Noah A. Smith. 2014. Tracking the de- velopment of media frames within and across policy issues.
Munmun De Choudhury. Shagun Jhaver, Benjamin Sugar, and Ingmar Weber. 2016. Social media partic- ipation in an activist movement for racial equality. In Proceedings of the international aaai conference on web and social media, volume 10, pages 92-101.
William J Brady, Julian A Wills, John Jost, Joshua Tucker, and Jay J Van Bavel. 2017. Emotion shapes the diffusion of moralized content in social networks. Proceedings of the National Academy of Sciences, 1 114(28):7313-3311
Morteza Dehghani, Kenji Sagae, Sonya Sachdeva, and Jonathan Gratch. 2014. Analyzing political rhetoric in conservative and liberal weblogs related to the construction of the "ground zero mosque". Journal of Information Technology & Politics, 11(1):1-14.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
Dorottya Demszky, Nikhil Garg, Rob Voigt, James Zou, Jesse Shapiro, Matthew Gentzkow, and Dan Juraf- sky. 2019. Analyzing polarization in social media: Method and application to tweets on 21 mass shoot- ings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putatiioaa Linguistics: Human Language Technolo- gies, Volume (Long and Short Papers). pages 2970- 3005.
Clinton Burfoot, Steven Bird, and Timothy Baldwin. 2.11. Collective classification of congressional floor- debate transcripts. In Proceedings of the 49th An- nual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1506-1515, Portland, Oregon, USA. Association for Computational Linguistics.
Dallas Card, Amber E. Boydstun, Justin H. Gross, Philip Resnik, and Noah A. Smith. 2015. The media frames corpus: Annotations of frames across issues. In Pro- ceedings of the 53rd Annual Meeting of the Asso- ciation for Computational Linguistics and the 7th International Joint Conference on Natural Language
Lingiaa Deng and Janyce Wiebe. 2015. Joint prediction for entity/event- weitt//veentleeell sentiment analysis using proba- bilistic soft logic models. In Proceedings of the 2015 Conference Empirical Methods in Natural Lan- guage Processing, pages 179-189, Lisbon, Portugal. Association for Computational Linguistics.

---

Zackary Okun Dunivin, Harry Yaojun Yan, Jelani Ince, and Fabio Rojas. 2022. Black lives matter protests shift public discourse. Proceedings of the National Academy of Sciences, I (1910))=211730)) 173201 19.
Anjalie Field. Chan Young Park, Antonio Theophilo. Jamelle Watson-Daniels, and Yulia Tsvetkov. 2022. An analysis of emotions and the prominence of positivity in# blacklivesmatter tweets. Pro- ceedings of the National Academy of Sciences. 11965)xe2055777119
Javid Ebrahimi, Dejing Dou, and Daniel Lowd. 2016. Weakly supervised tweet stance classification by re- lational bootstrapping. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 1012-1017.
Anjalie Field and Yulia Tsvetkov. 2019. Entity-centric contextual affective analysis. In 57th Annual Meet- ing of the Association for Computational Linguistics (ACL 2019).
Johannes C Eichstaedt, Garrick T Sherman, Salvatore Giorgi, Steven O Roberts, Megan E Reynolds, Lyle Ungar, and Sharath Chandra Guntuku. 2021. The emotional and mental health impact of the mur- der of george floyd on the us population. Pro- ceedings of the National Academy of Sciences, 11 118(39):e2 109139118..11
Deen Freelon. Charlton Mcllwain, and Meredith Clark. 2018. Quantifying the power and consequences of so- cial media protest. new media & society, 20(3):990- 1011.
Dean Fulgoni, Jordan Carpenter, Lyle Ungar, and Daniel Preotiuc-Piete Pisituc 2016. An empirical exploration of moral foundations theory in partisan news sources. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC" 16), pages 3730-3736, Portorož. Slovenia. European Lan- guage Resources Association (ELRA).
Robert M Entman. 1993. Framing: Toward clarification of a fractured paradigm. Journal of communication, 43(4)551-58.
Yotam Eshel, Noam Cohen, Kira Radinsky, Shaul Markovitch, Ikuya Yamada, and Omer Levy. 2017. Named entity disambiguation for noisy text. In Pro- ceedings of the 21 Conference Computational Natural Language Learning (CoNLL 2017), pages 58-68, Vancouver, Canada. Association for Compu- tational Linguistics.
Ryan J Gallagher, Andrew J Reagan, Christopher M Danforth, and Peter Sheridan Dodds. 2018 Di- vergent discourse between protests and counter- protests:# blacklivesmatter and#f alllivesmatter. PloS one, 13(4)ee019564.
Lisa Fan, Marshall White, Eva Sharma, Ruisi Su, Pra- fulla Kumar Choubey, Ruihong Huang, and Lu Wang. 2019. In plain sight: Media bias through the lens of factual reporting. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6343-6349, Hong Kong, China, Association for Computational Linguistics.
Octavian -ugen Ganea and Thomas Hofmann. 2017. Deep joint entity disambiguation with local neural attention. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2619-2629, Copenhagen, Denmark. Associa- tion for Computational Linguistics.
Salvatore Giorgi, Sharath Chandra Guntuku, McKenzie Himelein- Wachowiak, Amy Kwarteng, Sy Hwang. Muhammad Rahman, and Brenda Curtis. 2022. Twit- corpus of the# blacklivesmatter movement and counter protests: 2013 to 2021. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pages 12351
Shangbin Feng. Zhaoxuan Tan, Zilong Chen, Ningnan Wang, Peisheng Yu, Qinghua Zheng, Xiaojun Chang, and Minnan Luo. 2022. PAR: Political actor rep- resentation learning with social context and expert knowledge. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 12022-12036, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Salvatore Giorgi, Lyle Ungar, and H. Andrew Schwartz. 2021a. Characterizing social spambots by their hu- man traits. In Findings of the Association for Com- putational Linguistics: ACI-IJCNLP 2021, pages 5148-5158, Online. Association for Computational Linguistics.
Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chan- dar, Soroush Vosoughi, Teruko Mitamura, and Ed- uard Hovy. 2021. survey of data augmentation approaches for nip. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 968-988.
Salvatore Giorgi. Vanni Zavarella, Hristo Tanev, Nico- las Stefanovitch, Sy Hwang, Hansi Hettiarachchi, Tharindu Ranasinghe, Vivek Kalyan, Paul Tan, Shaun Tan, Martin Andrews. Tiancheng Hu, Niklas Stoehr. Francesco Ignazio Re, Daniel Vegh, Dennis Atzen- hofer, Brenda Curtis, and Ali Hürricttgluu. 2021b. Discovering black lives matter events in the United States: Shared task 3. CASE 2021. In Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2021), pages 218- 218-227, Online. Associa- tion for Computational Linguistics.
Anjalie Field, Doron Kliger, Shuly Wintner, Jennifer Pan, Dan Jurafsky, and Yulia Tsvetkov. 2018. Fram- ing and agenda-setting in Russian news: a computa- tional analysis of intricate political strategies. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pages 3570- 3580, Brussels. Belgium. Association for Computa- tional Linguistics.

---

Gabriel Grill. 2021. Future protest made risky: Ex- amining social media based civil unrest prediction research and products. Computer Supported Cooper- arive Work (CSCW), 30(5-6):11-833,
Volume / (Long Papers), pages 1875-1885, New Or- leans, Louisiana. Association for Computational Lin- guistics.
Kristen Johnson and Dan Goldwasser. 2016a. Identify- ing stance by analyzing political discourse on twitter. In Proceedings of the First Workshop on NLP and Computational Social Science, pages 66-75.
Suchin Gururangan, Ana Marasovie, Swabha Swayandipta, Kyle Lo. Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't t stop pretraining: Adapt language models domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342-8360, Online. Association for Computational Linguistics.
Kristen Johnson and Dan Goldwasser. 2016b. "all know about politics is what read in twitter": Weakly supervised models for extracting politicians' stances from twitter. In Proceedings of COLING 2016, the 26th International Conference on Computational Lin- guistics: Technical Papers, pages 2966-2977.
Jonathan Haidt and Jesse Graham. 2007. When morality opposes justice: Conservatives have moral intuitions that liberals may not recognize. Social Justice Re- search, 20(1) 1196
Kristen Johnson and Dan Goldwasser. 2018. Classifi- cation of moral foundations in microblog political discourse. In Proceedings of the 56th annual meet- ing of the association for computational linguistics (volume 1: long papers), pages 720-730.
Jonathan Haidt and Craig Joseph. 2004. Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus, 133(4):55-66)
Kazi Saidul Hasan and Vincent Ng. 2013. Stance clas- sification of ideological debates: Data, models, fea- tures, and constraints. In Proceedings of the Sixth International Joint Conference Natural Language Processing, pages 1348 1356, Nagoya, Japan. Asian Federation of Natural Language Processing.
Kate Keih, Itai Himelhoim, and Jeong-Year Yeob Han. 2018. Important tweets matter: Predicting retweets in the# blacklivesmatter talk on twitter. Computers in human behavior, 85:106-115
Katherine Keith, Abram Handler, Michael Pinkham, Cara Magliozzi, Joshua McDuffie, and Brendan O'Connor. 2017. Identifying civilians killed by po- lice with distantly supervised entity-event extraction. In Proceedings of the 2017 Conference Empiri- cal Methods Natural Language Processing, pages 1547-1557, Copenhagen, Denmark. Association for Computational Linguistics.
Kazi Saidul Hasan and Vincent Ng. 2014. Why are you taking this stance? identifying and classifying reasons in ideological debates. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 751-762.
Pamela M Hong and Clayton D Peoples. 2021. The ties that mobilize us: Networks, intergroup contact, and participation in the black lives matter move- ment. Analyses of Social Issues and Public Policy, 21(1):541-556.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representa- tions.
Joe Hoover, Gwenyth Portillo-Wighmms Leigh Yeh, Shreya Havaldar, Aida Mostafazadeh Davani, Ying Lin, Brendan Kennedy, Mohammad Atari, Zahra Kamel, Madelyn Mendlen, al. 2020. Moral foun- dations twitter corpus: A collection of 35k tweets annotated for moral sentiment. Social Psychologicul and Personality Science, 105--00011
Roudabeh Kishi and Sam Jones. 2020. Demonstrations & political violence in America: New data for sum- mer 2020. Armed Conflict Location & Event Data Project.
Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic lations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Short Papers), pages 452-457, New Orleans, Louisiana. Association for Computa- tional Linguistics.
Juliana Menasce Horowitz and Gretchen Livingston. 2016. How americans view the black lives matter movement.
Jelani Ince, Fabio Rojas, and Clayton A Davis. 2017. The social media response black lives matter: How twitter users interact with black lives matter through hashtag use. Ethnic and racial studies, 40(1) 18:14 1830.
Dilek Küçük and Fazli Can. 2020. Stance detection: A survey. ACM Computing Surveys (CSUR), 53(1):1- 37.
Mohit lyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies,
Ashutosh Kumar, Kabir Ahuja, Raghuram Vadapalli, and Partha Talukdar. 2020. Syntax-guide controlled generation of paraphrases. Transactions of the Asso- ciation for Computational Linguistics, 8:330-345.

---

Ashutosh Kumar. Satwik Bhattamishra, Manik Bhan- dari. and Partha Talukdar. 2019. Submodular uptimization-basee diverse paraphrasing and its ef- fectiveness in data augmentation. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguisties: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3609-3619, Minneapolis, Min- nesota. Association for Computational Linguistics.
SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop Semantic Evaluation (SemEval-2016)) pages 31 41, San Diego California. Association for Computa- tional Linguistics.
Marcia Mundt, Karen Ross, and Charla M Burnett. 2018. Scaling social movements through social media: The case of black lives matter. Social Media+ Society, ((4):2056305118800) 18807911.
Chang Li and Dan Goldwasser. 2019. Encoding so- cial information with graph convolutional networks forPolitical perspective detection news media. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics. pages 2594- 2604, Florence, Italy. Association for Computational Linguistics.
Nathan Ng. Kyunghyun Cho. and Marzyeh Ghassemi. 2020. SSMBA: Self-supervised manifold based data augmentation for improving out-of-domain robust- ness. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1268-1283, Online. Association for Computational Linguistics.
Chang Li, Aldo Porco, and Dan Goldwasser. 2018. Structured representation learning for online debate stance prediction. In Proceedings of the 27th Inter- national Conference on Computational Linguisties, pages 3728-3739, Santa Fe, New Mexico, USA. As- sociation for Computational Linguistics.
Minh Nguyen and Thien Huu Nguyen. 2018. Who is killed by police: Introducing supervised attention for hierarchical LSTMs. In Proceedingy of the 27th International Conference on Computational Linguis- tics, pages 2277-2287 Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. 2022. WANLI: Worker and AT collabora- tion for natural language inference dataset creation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages (82666647,, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min- Yen Kan. 2020. Fang: Leveraging social context for fake news detection using graph representation. In Proceedings of the 29th ACM in- ternational conference on information & knowledge management, pages 1165-1174.
Ruibo Liu, Guangxuan Xu, Chenyan Jia, Weicheng Ma. Lili Wang, and Soroush Vosoughi. 2020. Data boost: Text data augmentation through reinforcement learning guided conditional generation. In Proceed- ings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 9031-9041, Online. Association for Computational Linguistics.
Maria Leonor Pacheco, Tunazzina Islam, Monal Maha- jan, Andrey Shor, Ming Yin, Lyle Ungar, and Dan Goldwasser. 2022. holistic framework for analyz- ing the 19 vaccine debate. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 5821-5839, Seattle, United States. Association for Computational Linguistics.
Yinhan Liu, Myle Oit, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907
Kim Parker, Juliana Menasce Horowitz, and Monica Anderson. 2020. Amid protests, majorities across racial and ethnic groups express support for the black lives matter movement.
Nikhil Mehta. Maria Leonor Pacheco, and Dan Gold- wasser. 2022. Tackling fake news detection by con- tinually improving social context representations us- ing graph neural networks. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 1363-1380, Dublin, Ireland. Association for Compu- tational Linguistics.
Hao Peng, Ceren Budak, and Daniel M Romero. 2019. Event-driven analysis of crowd dynamics in the black lives matter online social movement. In The World Wide Web Conference, pages 3137-3143.
Dean Pomerleau and Delip Rao. 2017. Fake news chal- lenge stage I (fne-ii: Stance detection.
Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heinz, and Dan Roth. 2021. Re- cent advances in natural language processing via large pre-trained language models: A survey. arXiv preprint arxiv:2111.0123
Rajkumar Pujari and Dan Goldwasser, 2021. Under- standing polities via contextualized discourse pro- cessing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1353-1367, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Saif Mohammad, Svetlana Kiritchenko, Parinaz Sob- hani, Xiaodan Zhu, and Colin Cherry. 2016.

---

Lara Putnam, Erica Chenoweth, and Jeremy Pressman. 2020. The floyd protests are the broadest in us his- tory-and are spreading to white, small-town amer- ica. Washington Post, 6:2010.
Leo Graiden Stewart, Ahmer Arif, A Conrad Nied, Emma S Spiro, and Kate Starbird. 2017. Draw- ing the lines of contention: Networked frame con- tests within# blacklivesmatter discourse. Proceed- ings of the ACM on Human-Compule Interaction, I(CSCW):1-33.
Husam Quteineh, Spyridon Samothrakis, and Richard Sutcliffe. 2020. Textual data augmentation for effi- cient active learning on tiny datasets. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7400-7416 Online. Association for Computational Linguistics.
Qingying Sun, Zhongging Wang. Qiaoming Zhu, and Guodong Zhou. 2018. Stance detection with hierar- chical attention network. In Proceedings of the 27th International Conference Computational Linguis- rics, pages 2399-2409, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 188):
Robert F Tate. 1954, Correlation between a discrete and continuous variable. point- biserial correlation. The Annals of mathematical statistics, 2533):603-607.
Hannah Rashkin, Sameer Singh, and Yejin Choi. 2016. Connotation frames: A data-driven investigation. In Proceedings of the 54th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 311 1-112..
Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. arXiv preprint cx/0607062.
Oren Tsur, Dan Calacci, and David Lazer. 2015. A frame mind: Using statistical models for detec- tion of framing and agenda setting campaigns. In Proceedings of the 53rd Annual Meeting of the As- sociation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pages 1629- 1638, Beijing, China. Association for Computational Linguistics.
Shamik Roy and Dan Goldwasser. 2020. Weakly su- pervised learning of nuanced frames for analyzing polarization in news media. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7698-7716 7716, Online. Association for Computational Linguistics.
Shamik Roy and Dan Goldwasser. 2021. Analysis of nuanced stances and sentiment towards entities of
US politicians through the lens of moral foundation theory. In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, pages 1-13, Online. Association for Compu- tational Linguistics.
Marlon Twyman, Brian C Keegan, and Aaron Shaw. 2017. Black lives matter in wikipedia: Collective memory and collaboration around online social move- ments. In Proceedings of the 2017 acm conference computer supported cooperative work and social computing, pages 1400-1412.
Shamik Roy. María Leonor Pacheco, and Dan Gold- wasser. 2021. Identifying morality frames in political tweets using relational learning. In Proceedings of the 2021 Conference Empirical Methods in Natu- ral Language Processing, pages 9939-9958.
Marilyn Walker, Pranav Anand, Rob Abbott, and Ricky Grant, 2012. Stance classification using dialogic properties of persuasion. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 592-596, Montréal, Canada. Association for Computational Linguisties.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem. Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolu- tional networks. In European semantic web confer- ence, pages 593-607. Springer.
Ben Wang and Aran Komatsuzaki. 2021. GPT-J- 6B: A 6 Billion Parameter Autoregressive Lan- guage Model. https://gh thub. comk.//o////igfflllz// mesh- transforrrr--aaaa
Swapna Somasundaran and Janyce Wiebe. 2010. Rec- ognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and genera- tion of emotion in text, pages 116-124.
Denise J Wilkins, Andrew G Livingstone, and Mark Levine. 2019. Whose tweets? the rhetorical fune- tions of social media use in developing the black lives matter movement. British Journal of Social Psychology, 5844)7886-885
Dhanya Sridhar, James Foulds, Bert Huang, Lise Getoor, and Marilyn Walker. 2015. Joint models of disagrec- ment and stance in online debate. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 116-125.
Vanessa Williamson, Kris-Stella Trump, and Kather- ine Levine Einstein. 2018. Black lives matter: Evi- dence that police-caused deaths predict protest actiy- ity. Perspectives on Politics, 16(2)4000-15..

---

Chang Xu, Cécile Paris, Surya Nepal, and Ross Sparks, 2018. Cross target stance classification with self- attention networks. In Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics (Volume 2: Short Pupers), pages 778-783, Melbourne, Australia. Association for Computational Linguistics.
al., 2021 where entities are labeled for sen- timents and roles. We consider the entity la- bels ("target of care/harm".. "target of fair- ness/cheating". "target of loyalty/betrayal" "target of sanctityddegradationn, "authority failing over", "authority justified over"], in this dataset, as "targets" and the rest "ac- tors". We obtain the contextualized embed- dings of entities in tweet using RoBERTa by taking the embeddings of the last layer. Then we use a fully connected layer identify the sentiment or role of the entity. We use 80% data as training and the rest as validation set. We stop training the model when the valida- tion accuracy does not improve for 3 consec- utive epochs. We use these trained classifiers infer the sentiment and role of entities in the #BLM corpus. The validation accuracies of both of these classifiers on out-of-domain validation sets were 92%.
Guobin Yang. 2016. Narrative agency hashtag ac- tivism: The case offf blacklivesmatter. Media and communication, 4(4):13.
Yi Yang and Jacob Eisenstein. 2017. Overcoming lan- guage variation in sentiment analysis with social at- tention. Transactions of the Association for Compu- national Linguistics, 5:295-307.
Yiben Yang. Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang. Chandra Bhagavatula, Yejin Choi, and Doug Downey. 2020. Generative data augmentation for common- sense reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1008-1025, Online, Association for Computational Linguisties.
Caleb Ziems and Diyi Yang. 2021. To protect and to serve? analyzing entity-centric framing of police violence. In Findings of the Association for Compu- lational Linguistics: EMNLP 2021, pages 957-976, Punta Cana, Dominican Republic. Association for Computational Linguistics.
After obtaining entities, their corresponding roles, and sentiments towards them, we con- struct perspectives as such as "ppolice_positiv__ee actoor".. Now, we obtain Pointwise Mutual Information score (PMT) (Church and Hanks, 1990) for each per- spectives with the "pro-blacklivesmatee" and "ipo-bluelieesmatter"" stances using the follow- ing formula. For a perspective x we calcu- late the Pointwise Mutual Information (PMT) with a stance [(x, 8) using the following formula.
A Identification of Abstract Entities and Corresponding Perspectives
To identify the most common high-level abstract entities or the main actors in the pro-BlackLM and Pro-BlueIM tweets we follow the below steps.
We first determine the hashtags that are mostly used in the context of the keywords "black- livesmatter", "bluelivesmatter" and we assign soft labels of bluelivesmatter" to the tweets containing these hashtags, respectively. The identified "pro- blacklivesmatter and proo-bluelivesmatter" hashtags are shown in Table 8.
[(x,8) = loog P(x|s)
P(x)
Where P(z|s) is computed by taking all perspectives used in tweets with stance s and computing
and "pro-
and sim- ilarly, P(x) is computed by counting perspec- tive x over all of the tweets. We discard all perspectives that appear less than 0.5% of the time in the whole corpus.
count(x)
We extract noun phrases in these tweets using SpaCy. We treat these noun phrases entities. We remove all entities that are pronouns for this analysis.
We manually go through the high-PMI per- spectives with each stance and cluster them to form abstract entities and perspectives. The high-PMI perspectives are shown in Table 8. We cluster entities and perspectives that are consistent and directed to the same set of en- tities. For example, in pro-bluelivesmatter tweets, "a poot_neg_acte and "# blacklives- matter grrests__eg__att are directed to
We build two separate RoBERTa-baud (Liu et al., 2019) classifiers identify sentiment to- wards entities (positive/negative)) and the role of the entities (actor/target). These RoBERTa classifiers are trained using out-of-domain data. We use the annotated dataset by Roy

---

the same high-level entity "BLM Movement" and express the same perspectives. Hence, we merge these entities to the abstract en- tity "BLM Movement" and identify the rel- evant perspective towards this entity in pro- bluelivesmatter as "Negative Actor". The de- scriptions of all identified abstract entities can be found in Table 9.
step for each (stance, abstract entity, entity perspec- tive) tuple in Table 2. For example, to generate tweets that are "pro-BlackL.M" and portray "Black Americans" as "positive target", we prompt GPT-3 by using the prompt shown in Figure 3. We find that GPT-3 rarely generates repetitive tweets. We prompt GPT-3 multiple times for generating at least 20 unique examples per (stance, abstract entity, en- tity perspective) tuple. One author of this paper skims through the GPT-3 generated examples detect any inconsistency or unwanted bias in the generations and discards the generation if found any. The observation is that the GPT-3 genera- tions are mostly clean. We use OpenAl interface? for generating the tweets using text-davinci-003 (largest available) version of GPT-3 (till January 2023) using the following hyperparameters: top- p=1, frequency penalty: 1, presence penalty=1, temperature=1, max len=500. Note that, on the OpenAl console usage of GPT-3 for any experi- ment is a paid service. All of the generations used in this paper cost us in total ~ $3 US Dollars.
B Prompting LLMs to Generate Few Training Tweets
this section, we describe the details of few train- ing data generation using Large Language Mod- els (LLMs). We conduct our initial experiments with two LLMs GPT-3 (175B parameters) (Brown et al., 2020) and GPT-J (6B parameters) (Wang and Komatsuzaki, 2021) 1) and find that GPT-J generates a lot of repetitive examples. We conjecture GPT-J being a very small LLM in terms of parameters the main reason for a less diverse generation. This is line with recent findings where larger LLMs were found to perform better in various tasks than the smaller LLMs. Hence, we retain only the GPT- 3 generations in this paper. The generation process using GPT-3 is described below.
B.2 Preprocessing and Labeling of Generated Tweets
After the generation of tweets using GPT-3 that contain the elements from (stance, abstract en- tity, entity perspective) tuples, we preprocess these tweets identify the text span containing the ab- stract entities. Note that, an abstract entity can be addressed differently in different tweets. For example, "police" can be addressed as "cops" or "law enforcement". To identify the abstract entity containing spans in generated tweets, first run SpaCy noun phrase extractor these tweets and automatically group the extracted entities based on keyword match, for example, "police" and "po- lice force" will be grouped together because the common keyword "police". Then an author of this paper looks at the entity groups and discards entity groups that are not related to the target ab- stract entity. Then we consider the rest of the entity groups and annotate them gold data for abstract entity mapping. The statistics of generated data are shown in Table 10. Two examples of GPT-3 generated tweets for each perspective are shown in Table 11. We submit a randomly selected subset (20 tweets) of the GPT-3 generated annotated data with this manuscript for review. Upon acceptance of this paper, we will release the whole set.
B.1 Generation
For generating a few tweets containing the stances and corresponding entity perspectives (as described in Table 2), we prompt the GPT-3 model (Brown et al., 2020) using the prompt structure shown in Figure 7. We prompt GPT-3 in such a way that all of the structured elements for a stance and corre- sponding perspective are present in the generated tweets. To ensure that, shown in Figure 7, we in- struct GPT-3 model to generate 30 different tweets that must fulfiil the following three conditions.
(1) Explicitly mention one or more entities from the following list centity list> and portray them as <entity perspective>.
(2) Be in the support of the <movement name> movement.
(3) Use one or more hashtags from the following list <hashtag listz.
Here, <entity perspective> can be one of the following "positive target", "positive actor", "negative actor" and <movement name> is either "#BlueLivesMatter"" We take the <entity list and <hashtag list> from Table 8. We perform a separate prompting
Phttps://bett openai. cotpm/aiiimmmppaaruuuddn

---

Stances
Top 20 Most used hashtags
High PMI perspectives
"the activists pos_ target'. *# ahmaudarbery pos target', '# breonnataylor_ pos, target', 'a petition_ cattiio_pos_taagerr,, '# jus- lice__os_actoo
#blacklivesmatter. #bim, #gcorgefloyd. #alllivesmaterr #policebrutality. #justiceforge- orgefloyd, #dcfundthepolice. #racism, #nojusticenopeace, #covid19, #icanthreathe, #bre- onnataylor, #bluelivesmatter, Mprotests2020, #antifa, Wtrump, #hlackliiesmatterukk, #police- brutalitypandemie, #protests, #georgefloydprooesst
deproteets__pon Bacss_oos_taargee poossagtttt pne_pos_targee *# black-
*# justice_poss pos_tagee "# georgefloyd__oo_rrree 'a black man_pos_larget
livesmatter # protests_pos_targee 'a while mam momm_pos_aarget" '# amer- ica pos actor'. '# rayshardbrooks pos, pos_ttere'' "# blacklives- matter # protests, pos, actor'. "african americans pos target', 'the kkk neg actor', "goverament neg actor', whitepriv- ilege neg, neg_actot actor'.
prreddom_ossstrrge
Pro Blacklivesmatter
whitesupremacy neg_ actor'. "a white man man_neg_atte pee_eneg_ellee _cceerte"" '# racism_ neg actor,, , buebiden_neg_aclue '# trump neg actor'. ica neg actor'. '# policeviolence neg actor'. 'republi- cans neg actor", 'media_ neg actor', 'president neg_actor" actor'
black man neg actor', '#
Hbluelivesmatter, #alllivesmat- ter, #blacklivesmatter, #backthe- blue. #whitelivesmaater. #maga, #trump2020, #bluelivesmatters, #maga2020, #blm, #thinblueline, wwhhttouutwednesdayy #women- fortrump, Wkag, #buildthewall, Halllieesmaters, #blueflu. #po- lice. #defundthepolice, #lawen- forcement
pooss_oo__rrreetttteeep target", "# cops_pos____..oppppppppooooesess pos_actor', jaw_pos___r targttt""" 'repub- lieans ppos_psss___e target', 'god_pos__te good_pos_actor', sus_acttr', 'communities Bacliiesspps_______ pos_target' '# Botiiddonn_osstargett 'the country_pos_tane siossttret', , '# equal- '# america_pos_tareee pos_eet 'a niol_neg_actor,, '# democrats _et__en____e__eee aet_acto', blacklivesmatter protests_nne _at__nn________ actorr, ppoliticians_nege
Pro Bluelivesmatter
'# antifa_ neg_actorr", ; '# cops_neg_actor", "# obama_nege @hammanne_actor"
Table 8: Data analysis for abstract entity and corresponding perspectives identification.
Abstract Entilies
Description
and the baselines that are trained with the ar- tificially crafted data vs. the real annotated data.
Black Americans
Refers Block Americans.
Refers
depertment onfonoement
Community
Refens the eniiiies that refer communal spint such commu- niies, sociity nation, United Stiies of America, citixens, etc.
Racism
Refir racsom and racists.
The human annotation of real data occurs in the following three steps.
Refer Democratic party the politicians this
party or anyoee supporting them.
Republicans
Refer to the Republican pony in the USA, politicios from this supporting them.
Step-1: From the whole #BLM corpus summa- rized in Table 1, we rank authors who are most consistent in the usage of the keywords blacklives- matter and bluelivesmatter. The most consistent user uses only either of the keywords 100% of the time. Then we randomly sample 100 users from the top 500 consistent users of blacklivesmatter and bluelivesmatter resulting in 200 users. We annotate these users for their stance mro-blaackiivesmatee pro-blueiivesmatter We present human evalu- ators with each author's id, their profile description, and all of the tweets they shared. Note that the au- thor names were hidden and the author ids were just numeric ids assigned by Twitter to Twitter users. Then we ask the human annotators to annotate the authors for stance by looking at their profile de- scriptions and the tweets they shared. The human annotators asked to annotate 'none" if it is not possible infer the stance of an author by looking at the tweets and the profile description. We find an average anteratno agreement of 0.824 (almost perfect agreement) using Cohen's Kappa measure.
suthuritatioo figures president, governors, mayess, and on.
Refer government
White Americans
Refas to White Americans.
BLM Movement
Refer to the Blackl avsouittrr muvemeni, the protesturs, the supponers
Refers any ofticial petiton or campsign for uupprtiin a cuuse
Autifa
The Anti Pascist and Auz. Racist politcall movemets the USA.
Table 9: Abstract entities and their descriptions.
C Real Data Annotation Procedure
We numan-annotall a subset of the whole #BLM corpus as described in Table for the following two purposes.
To evaluate the performance of our proposed model on real data and compare it with the baselines.
evaluate the quality of the artificially crafted training data (using LLMs) by compar- ing the performance of our proposed model

---

iprompt-start
Write 30 different tweets that must
Explicitly mention one or entities from the following list <entity list> and portray them as centity perspective>.
in the support of the <movement name> movement.
3, one or more hashtags from following list: <hashtag list>.
Answer:
<prompt-end>
stenetation-start>
1. <generated tweet>
2. <generated tweet>
3. <generated tweet>
Figure 7: Prompt structure for GPT-3 to generate tweets containing specific stances and perspectives. The black-
colored text is the imput text in the prompt and the blue-colored texts are generated tweets by GPT-3.
Generated tweets having entity mention
Generated tweets
Stance: Perspective
# Entities
roobblacklivesmatter: whites neg
27
41
moo-backlivesmatter blm movement pos target
30
28
28
blllkkkiiiesmtttrr: petition pos target pro-blacklivesmatter police neg actor pro
30
30
36
33
31
bl-bbllliieematterr blm movement pos actor
30
30
30
noobblacklivesmatterr govermment neg actor pro-placcee
30
28
35
bl-bballiivemaaterr blacks pos target
40
30
30
noobblacklivesmatter: racism neg actor pro-
29
24
27
nro-bluelivesmaater: community pos target
28
34
pro- nrobbluelivesmatter; police pos actor mro-bluelivesmatter: antifa neg actor pro
36
34
28
29
27
30
noobbluelivesmatte:: police pos target
33
30
pro-bluelivesmatter: blm movemen actor pro bluelliisssmtter: republicans pos actor pro-bluelivesmatter: blacks neg actor pro- boobluellivesmater: democrats neg actor
30
30
24
28
47
40
40
40
30
26
26
Total
582
476
517
Table 10: GPT-3 generated training data statistics.
The disagrecments between the two annotators are resolved by discussion. After annotating authors for stances, we readily get stance labels for the tweets they wrote. We find that often the supporters of the movements use the keywords/nashtags that are used mostly by the movemeett to troll or criticize the opponent. This is in line with previous findings computational social science (Gallagher et al., 2018). We annotate these tweets as "Am- biguous Tweets". Ambiguous tweets are intuitively more difficult for models to disambiguate because they use the keywords and hashtags that are fre- quently used by the opponents. We determine tweet to be ambiguous tweet if it is annotated by human annotators "pro-blacklivesmatte" but
the tweet uses the keyword "bluclivesmatter" and vice versa for "pro-bluelivesmatte"" Examples of ambiguous tweets are presented in Table 12.
Step-2: After annotating tweets and authors for stances we extract entities from these tweets us- ing SpaCy noun phrase extractor and in this step, we annotate these entities for abstract entity labels. We present two human annotators with a tweet, its stance (annotated in the previous step), and an entity mentioned in the tweet. Then we ask the annotators assign abstract entity label the entity from the list of abstract entities in Table 2. The annotators could select multiple abstract entity labels or "none" for an entity. We find an average Cohen's Kappa annotaar agreement score

---

Stance: Perspective
GPT-3 Generated Tweets
1. We must continue to fight for justice and equality for all black people. dulllivesmatter Apolice
pro-blacktlvemmaater: blacks pos target
bottalityanndemie
2. We will no longer tolerate the unjust murder of black people by those in pusitions of power. #bia apolicebouiallyyunddeee
Black Matter protesis been etlectivee in bringing awareness to police brutality pandemic. thlacklivesmatter thim
pro-blacktiresmatter: blm movement pos actor
The Black Lives Matter movement is helping to raise awaremess about the issues lacod by minorities. #blacklivesmatter
I stadd with the BBlackLivvssMatee movement and prosest against policc brutality! #policebrutal-
pro-hlackivesmatter: blm movement pos target
ity thlm
the fight to end police beutality and racism by supponing MBlackLivesMaan movementt!
We make a difference by comning together and speaking out against these injustioes!
1. From Ferguson to Raltimore, it's clear that the government doesn't care about us. Time for chunge! Holacklivesmatter
pro-blackdi government neg actor
2. The government has blood their lunds and won't rest until they gone! #blacklivesmateer #alm
Sign the petition hold police accountable for their brstality! #blacklivesmatter
pro-blackthvesmatter: petition pos target
Show your support and sign the petition now? Halliivemstterr
1. The police are supposed to protect us, not kill us. How many more innocent lives will be lost before something is done? apolisetrutaliyyaaddeme Police brutality must end now! Outrage is enough, we demand action! APoliceBruutaliyysa damic
pro-blacktivesmatter: police neg actor
1. Racism, like police brutality, is a systemic problem that needs to be sudresssed immedialely. #nlacklivesmatter shim
pro-blacklivemaatte:: racksm neg actor
2. Silence is complicity! If' you' te not purt of the solution. you're part of the problem. Show your suppori and Stand up aguinst bigotry! #blacklivesmmller
4w hiteprivilleee is real it's something that needs to acknowledged and dismantled. No one should have power over another because of the color of their skin
pro-hlactivvemmatter: whites neg actor
It's time to take stand against the hate and bigotry of #whitesupeenucisss We must come
together and show we are committed cquality and justice for all.
Antifa domestic temorists are raining our country! #bluclivesmatter Halliivesmnter
pro-bludivessnatter: antifa neg actor
stand with the hrave men and women of law enforcement who keep our country safe from
groups like antifal #hackthsblue
1. It's always the black people who cause the most trouble in our community. dalliieesmatter #eleelivesmatter #hacktheblie
pro-blnclivesmatter:: blacks neg actor
can't It irust black people to obey the Hwhiteeivesmmatlrr #allivesmutter Rbacklleblue
1. The blacklivesmatter protests nothing but excuse to riol and loot! #bluelivemmater
pro-bluelivesnnatte::: blm movement neg actor
Halliieamatter
BLM protests put inmacent at risk! Holuelivesmatter
We stand with our communities and the police who keep safe. #bluelivesmatter Abacktheblue
pro-blnelivesmatter: community pas target
2. I'll never understand why people want to harm police officers who are just trying to do their job and keep our communitic ities safe. PBlueLLivsMaare
The left is always championing criminals and trying down our police officers. #hinelives-
pro-blueivesauaterr democrafs neg actor
multer Roucktheblue
Dems don't care about and order, they only care about chans and anarchy. #bluelliesmatee
Police put their
the every day protect #blacklivesmaterr
pro-blueiiessmateer police pos actor
2. The media portrays law enforcement in negative light but I know they are doing an amazing
job!!! #ProudOrCurPolice
Thank you
and women enforeement who keep us safe! #biuclivesmar-
pro-blnelivesmaater: pollic pos target
ter #hacktheblue
I siand with the police and oppose the violence against them! RAlltivvsMaatte
stunding behind all police officers they work make our communities safer let's keep on huilding an America where blue lives matter! suhitelivemmnter The Regublican parry stands fimmly hehind those who serve and protect us each day
ure proud comservatives
pro-hlnelreesmatter:: repnhlicans pos actor
essential scknowlelge lheir efforts and reject any altempts to discredit ihem. #bluelivematters
Table Il:
Examples of GPT-3 generated tweets.
0.697 (substantial agreement) for this task. The disagreements are resolved by discussion. We find that 78% of the entities map to at least one abstract entity and a small portion of entities (<5%) are identified to map to multiple abstract entities. It im- plies that the abstract entities summarized in Table 2 are the main actors related to the movements and they have very good coverage of the whole data.
Based on the feedback from the human annotators, the unmapped entities are mostly cases where it is difficult to determine what abstract entity they are referring to without more context and a small portion of them were wrong entity detection by the SpaCy noun phrase extractor.
Step-3: In this step, we annotate the entities annotated for abstract entity labels in Step-2, for

---

Ambiguous Tweets
Annotated Stance
SPAMMING RACIST TAGS #Qanán #0ru #tump #EXPOSEANTIS #ExposeAntifaTer
rorists #bluelivesmatter #Whitclivesmatter #whhicelivesmattiernnss #tt.://eeroonoosooioooooeeeeee #whhiccdiiessmattemmrr #white- livesmattertoo WWWGIWC WGA
pro- rrobblacklivsmutter
course WivteMiivsMaatter mailmen are the backbone of the nation
pro- pro-blacklivesmatter
Tell about the good cops #BLUEFALL #Bluel Materr
pro noobblackiivemmateer
Mateers protestors are racist cop killers and vandals Not FoxNews @ Twitter should remove the lying hashtag Newisssssaa as these thugs vandalize memorials of war heros on #DDuy
pro-bluelivesmatter
#BLM Burn. Loot. Murder. Matee is joke.
pro-bluelivesmatter
#BlackLiveMMatter. ... They really don't, to the black lives group! They must have gotten their group minutes from #Metoomovement #LIARS
pro-bluelivesmatter
Table 12: Examples of tweets and their annaattadnttted stances where supporters of a movement use hash- tags/keywords related to the counter-movement to criticize or troll them. We annotate these tweets ambiguous tweets.
sentiments toward them (positiveenegatiiv)) and as- signed roles (actor/target). We present two human annotators with the tweet text, its stance, entity mentioned in the tweet, and the abstract entity label of the entity (all determined in the previous two steps). Then we ask them to annotate the entity for role and sentiment. For role identification, the annotators are instructed to select "none" if it is not clear from the tweet text what the role of the entity is or select "both" if the entity is portrayed both actor and target. For example, in the tweet "Police keep us safe. We should defend our Police.". the entity "Police" is portrayed be both posi- tive actor and positive target. We find Cohen's Kappa inter-annotator agreement scores of 0.976 and 0.815 for entity sentiment and entity role an- notation, respectively, that are almost perfect. We resolve the disagreements by discussion. In < 1% cases, we find an entity to have both the actor and target roles in tweet.
from the annotated dataset as all the models we trained are trained to predict one class and intu- itively these cases difficult for even humans disambiguate. In the entity mapping annotation, we find < 5% entities that map to multiple abstract en- tities. We randomly select one abstract entity from the multiple abstract entity labels as the final label. It results in 189 users annotated for stance, 2, 980 tweets annotated for stance (520 among them are annotated as ambiguous tweets), and 2, 091 enti- ties annotated for abstract entity labels, sentiment toward them, and their roles.
Human-Annotated Training and Test Data Se- lection: We randomly sample 50 authors from the 189 annotated authors. These 50 authors, their tweets, and the entities mentioned in those tweets are defined as human-annotated real training data. We define the rest of the annotated dataset as numan--nnotalle test data. Our proposed model and the baselines are trained in the weakly super- vised setting using the GPT3-generated training data and in the directly supervised setting, they are trained using the human- annotated real data. In both cases, the models are tested on the human- annotated test dataset. The statistics of the GPT3- generated training set, the human-annotated train- ing set, and the Human-annotater test set are shown in Table 3.
The per-label agreement scores for each anno- tation task can be found in Table 13 and the final annotated data statisties be found in Tables 14 and 15.
Both of the human annotators graduate stu- dents (age above 21) and they were awarded re- search credits for this annotation task. They were sufficiently briefed on the tasks and were informed that the tweets may contain potentially sensitive language. The annotators were also informed that the dataset will be used for research purposes.
D Experimental Setting
D.1 Preprocessing of data
Data Consolidation: As described above, for the entity role annotation we find < 1% entities that map to multiple labels. We discard these entities
We collect the tweet texts of the tweet ids provided in the source dataset (Giorgi et al., 2022) using

---

AUTHOR STANCE ANNOTATION
Total
#BlackLMM
#BlueL.M
Stance
Agreement
Pro #Blackl. ivesaarr
0.824
Authors
Maete
0.824
All Tweets
1943
1037
2980
Average
0.824
Amhiguous Tweets
314
206
520
ENTTTY MAPPING ANNOTATION
Table 15: Annotated data statisties for authors and tweets.
Abstract Entities
Agreement
antifa
0.530
blacks
0,834
movement
0,728
community
0.442
democrats
0.706
extract the entities from tweet text.
govermment
0.623
petition
To extract keywords from the author profile de- scriptions, first, identify all hashtags used in the description and add them the keywords list, then we extract nngaams S S 3) from the residual text. Then we merge each ngram to single word and check is similar to a hashtag (e.g., merged ngram "Black lives matter" is similar to "#black- livesmatter"). ) If it is similar we add that ngram to the keyword list. For the residual ngrams we look at the most occurring ones and manually discard those that do not imply any meaningful message. We add the of the ngrams to the keywords list.
police
0.913
rucism/rrcistt
0.553
repuhlicans
0.689
whites
0.653
Average
0.697
ENTITY SENTIMENT ANNOTATION
Sentiment
Agreement
Positive
0.976
Negative
0.976
Average
0,976
ENTITY ROLE ANNOTATION
Role
Agreement
Actor
0.782
Target
0.847
Each element summarized in Table corre- sponds unique node in our graph. For example, two tweets mention lexically equal entities, the entities will have two different node representations in the graph. Because the perspectives towards the lexically equal entities may be different in the two tweets. For example, "law enforcement" may be portrayed as "positive actor" in one tweet and "neg- ative actor" in another. Hence, the number of nodes and edges in our graph corresponds to the statistics in Table
Average
0.815
Table 13: Inter annotators agreement for the data an- notation process. Cohen's Kappa scores are used as agreement.
Abs. Entities
Count
Sentiment
Role
Neg
Actor
Target
police
784
464
320
439
280
whites
76
6
70
72
3
black-people
583
539
44
61
516
racism/racists
130
0
130
130
0
blm movement
286
198
88
158
30
democrats
111
7
104
111
govermmen
46
3
43
46
0
D.2 Task Adaptive Pretraining RoBERTa
republicans
79
45
communities
71
70
1
1
70
Following previous works (Gururangan et al., 2020), we perform task adaptive pretraining of RoBERTa for our task. We continue pretraining RoBERTa with the whole word masking technique. In this approach, we randomly select some words and mask the whole word. Then we predict the original vocabulary ID of the masked word based on the context it appears in. We use unused data of the #BLM corpus from (Giorgi al., 2022) in this pretraining step. We find that this task adaptive pretraining improves classification results signifi- cantly over simple RoBERTa as shown in Tables 4 and 16.
petition
0
antifa
91
7
84
91
0
Total
2268
2268
2091
Table 14: Annotated data statistics for entities.
Tware APT 3 Before using the #BLM corpus collected from (Giorgi et al., 2022), we remove all non-ASCII characters and URLs from the tweet text. We used SpaCy Noun Phrase Extractor to
http://warrrrppeeeeerrrraaaaecee latest/api/client

---

MOneEX
Weok Sep.
Weak
Dest
Weak
Dirtt
Weak
Divett
Week
Direet
Wook
Divect
NAIV
Kunsee
Koyoou Read
85
29 aal
B-r LAS
20.55
15.19 14:
B0LsS a 1.37
103.107
19.7
67.56
34.55
80.07 4-30
50.35 0.6
20.0
64,35-133
4
54.79
05.18 61
00.50-0 4:34
31:5411 15
54.1 1-18
15.55 885
79.7 4:37
15.0--15
51.77
01984:12
MELTITASK
9101 2.4.
01
72:22 60
550 1.3
565522 11
2:3
68.35
02:6--31
5807 2.
Astur Einbed
02.00 1/1
01.46
717
66.33
13
00.0
1:2
10.56
07.32
9:
01.90 121
OUR MODEL
AAtber Netwass
Gool
63.61
80.50
50.75 1.7
15
61.16+114
6653
4336
84
83:87
15.20 2.3
01
62:61:12
75.471.8
78.88 0.
Sort awiing
01.78-1.1
90.96-10
61. 0.4
66.00-25
64336-.11
00.0-677
871--0
67.72-04
67.14-06
87.05-003
07.03-24
73.35-15
Table 16: Average weighted Fl scores with standard deviations for classification tasks over 5 runs using 5 random seeds: 1000, 2000, 3000, 4000, 5000.
D.3 Our Model Initialization and Hyperparameters
label confidence threshold of 0.9 till epoch 200 and reduce it to 0.8 after that, as many training amples are already added till then and the model is intuitively more stable. We set the tweet threshold, t for author consistency check to 10, 5, and at epochs 1, 20, and 50, respectively. We find that the majority number of tweets (> 75%) become consistent and are added to the self-learned train- ing set till epoch 300. The hyperparameters are determined empirically.
We initialize the representation of each node type in our graph using this RoBERTa-tapt. Author nodes are initialized by RoBERTa- tapt embeddings of the author profile descriptions or average embedding of randomly sampled tweets by the author if no profile description is found.
We pretrain the external classifiers (Csent: Crote) on out-of-domain data proposed by Roy et al. (2021). In this dataset, entities mentioned in tweets from US politicians are annotated for their moral roles. The moral roles are associated with posi- tive and negative sentiments and actor and target types. As a result, we get annotated dataset for pre- training our external classifiers, Csent and Crotes for sentiment and role classifications, respectively. To train these classifiers, contextualized embed- dings of the entities are obtained using RoBERTa- tapt, and a fully-comnected layer is used to identify role/sentiment in these two classifiers. We train these classifiers until the accuracy in a held-out validation set (20% from OOD) do not improve for three consecutive epochs. The accuracies of both of these classifiers the out-of-domain validation sets were 92%. We stop backpropagating to RoBERTa-tapt when Csent and Crote are combined with our framework, however, the fully-connected layers are updated.
D.4 Baselines
Naive: We implement two naive baselines. The first one is a random selection where labels of tweet stance, entity sentiment, entity role, and entity map- ping are determined by a random selection of corre- sponding labels. We perform the random selection using five different random seeds and report the average results. The second naive baseline is ap- plicable for only tweet stance classification. We follow a keyword-matching approach similar to Giorgi et al., 2021 la for that. For the classification of tweet stance using keyword-matching, we use the keywords available with each data point in (Giorgi et al., 2021a). In this dataset, each tweet is marked with one more of the following three keywords [blacklivesmatter, bluclivesmatter, alllivesmatter] based on their presence in the tweet. We classify a tweet to be pro- pro-BlackLMM if it marked to have the keyword blacklivesmatter in it and label as pro- BlueLM if it contains bluelivesmatter. In case of a tie or not availability of any one of these keywords, we break it randomly. We do not consider the key- word alllivesmatter in this classification approach because this keyword is more ambiguous and used widely by both of the movements. We determine the author stances based on the majority voting on the identified stances of the tweets they wrote.
We use a 2-layer R-GCN to encode our graph nodes and 768d input node features are learned in 100d and 50d spaces in the 2 layers of the R-GCN. We use a learning rate of 0.0005 to train the model. We infer after every 10 steps and before the first inference step, we train the model until the total training loss does not increase for 3 consecutive epochs. We stop training the model if the number of new training examples is less than 0.3% for 10 consecutive inference steps or a maximum epoch of 300 is reached. For consistency check, we use
Discrete Text Classifiers: We implement the sec- ond type of baseline that are discrete text classifiers

---

based on pre-trained RoBERTa. We finetune sep- arate RoBERTa-based classifiers for each of the tweet stance, entity sentiment, entity role, and en- tity mapping classification tasks.
the last hidden layer is used as the tweet embedding and it is passed through a task-specific wwo-hidden- layer feed-forward neural network to predict the stance of the tweet.
For the classification of tweet stances, we encode the tweet text using Ro The represen- tation of the [CLS] token of the last hidden layer is used as the tweet embedding and it is passed through fully connected layer to predict the stance of the tweet. We update RoBERTa-tapt parameters as well during the learning steps. We stop learning when the validation accuracy does not improve for three consccutive epochs. The author stances are determined based on the majority voting on the identified stances of the tweets they wrote.
For the classification of entity roles, sentiments, and mapping, we encode the entity-mentioning seg- ments in the tweet texts using the shared RoBERTa- tapt encoder. We take the representation of the last layer and select tokens corresponding the entity spans in the tweet text. Then we average these selected tokens' embeddings to get the representa- tion of the entity. Then three different task-specific two-hidden feed-forward neural networks are used to predict the sentiment, role, and abstract entity mapping of the target entity, respectively. We define the multitask loss function, Las as follows.
For the classification of entity roles, sentiments, and mapping we encode the entity-mentioning seg- ments in the tweet texts using RoBERTa-tapt. We take the representation of the last layer and select tokens corresponding to the entity spans in the tweet text. Then we average these selected tokens' embeddings to get the representation of the entity, Then fully connected layer is used to predict ei- ther of sentiment, role, or abstract entity mapping of the target entity. We update RoBERTa-tapt pa- rameters as well during the learning steps. We stop learning when the validation accuracy does not im- prove for three consecutive epochs. To make the entity role and sentiment classification baselines comparable to our proposed model, during train- ing, combine the OOD data from (Roy al., 2021) with the LLM-gennrated training data or the training data in the weak and di- rect supervision settings, respectively.
LM Astonnnce + AzLssent + daalrrte + AsLmap
Here, Lstance is tweet stance classification loss, and Leent, Lrole and Lmap are entity sentiment, role, and mapping classification losses, respectively. All of the losses are Cross Entropy losses and we set,
A = A2 = A = A = for our experiments. We pretrain the shared RoBERTa- tapt and the sentiment and role classification-spel specific neural networks with the OOD data from (Roy al., 2021) in the same way we pretrain the Ceent and Crote classifiers that are used in our model. All the specifcc two hidden layers consisting of 300 and 100 neu- rons with ReLU activations. We keep training the shared RoBERTa-tapt and the task-specific clas- sifiers using either the artificial training data or real training data. We stop train- ing when the combined tweet stance, entity senti- ment, role, and mapping classification accuracy do not improve for five consecutive epochs.
neural networks use
Multitask: Our model jointly models perspec- tives with respect to entities (sentiment towards them, assigned role, abstract entity mapping) and the stances in the tweets. Context-rich represen- tations of entities and tweets are learned and the single unified representation is used to infer various labels such stance for tweets and, sentiment, role, and mapping for entities. As a result, the closest match to our model is the multitask approach.
Multitak with Author Information: The multi- task approach described above jointly models only textual features. However, our proposed model (as described Section 3), is able to incorporate author information and social interaction among them. Hence, for a fair comparison of the multitask baseline with our proposed model, we enhance the multitask baseline with author information.
To implement the multitask baseline, we define a single pre-trained RoBERTa-tapt text encoder that is shared across all classification tasks such tweet stance, entity sentiment, entity role, and entity mapping.
For the classification of tweet stances, we en- code the tweet text using the shared RoBERTa-tapt encoder. The representation of the [CLS] token of
First of all, we learn rich author embeddings using relational graph convolutional networks (R- GCN). For that, we create a graph consisting of

---

only author nodes and keyword nodes (as described in Section 3). Two authors are connected to each other with the retweet relationships and an author is connected to a keyword node if they use the keyword in their profile description (as described in Section 3). Then we learn the author embeddings in this graph using two-layer R-GCN with link prediction objective. In this learning objective, the similarity between two author nodes is maximized if they are connected in the graph and the similarity is minimized if they are not connected in the graph. The link prediction loss function, Liin is defined as follows.
specific feed-forward neural networks in the multi- task model. The rest of the multitask learning ob- jectives and the hyperparameters remain the same as described above. In this manner, the multitask approach with author embeddings becomes com- parable to our proposed model as they incorporate the same information (textual and author network) in the learning process.
D.S Variations of Our Model
We study our model in the following four variations as shown in Tables 4 and 16.
(1) Text-Discrete: In this version, only textual elements such as the tweets and the entities, are considered. The interactions among them are not modeled. For example, the nodes corresponding tweets and entities are not connected using edges. Just initialized node representations are used for classification. This version is comparable frozen RoBERTa.
Liink =1 -aim(a;, ap) + simmas, an)
Here, sim(a, ap) is the similarity between author embeddings at and ap, where author, t is connected to author, p in the graph. sim(ai, an) is the similar- ity between author embeddings at and an, where author, is not connected to author, in the graph. We measure and define similarity as the dot product between author embeddings. We randomly sam- ple 5 negative examples (two authors are not con- nected) for each positive example (two authors are connected) in each layer of R-GCN.
(2) Text as graph: In this variation, the text is converted to a graph consisting of tweet, entity, and hashtag nodes. The relations among these elements are modeled using edges. Note that no author infor- mation is added. Tweet stances are inferred only conditioning on learned tweet node embeddings. No self-inference is done in this variation. In this variation, train our model for at least 15 epochs and after 15 epochs stop training if the total loss does not increase for three consecutive epochs. (3) Text as graph + Author Network: In this variation the author network is added to the text- only graph, however, no self-learning is done. In this setting, we train our model for at least 15 epochs and after 15 epochs we stop training if the total loss does not increase for three consecutive epochs.
We train the R-GCN layers by optimizing the loss Link for 10 epochs. Author nodes are ini- tialized using the RoBERTa-tapt encodings of the author profile descriptions or average embedding of randomly sampled tweets by the author if no profile description found. The keyword nodes are initialized with their RoBERTa-tapt encodings. All author embeddings are learned in a 50 dimensional space using the 2-layer R-GCN. In this manner, we obtain rich 50 dimensional author embeddings that encode the author retweet relationships and their profile descriptions.
(4) Text as graph + Author Network + Self- Learning: This is the final version of our model where self-learning is added with the combined text and social graph. In this version, we follow the stopping criteria as described in Section D.3.
We enhance the multitask baseline described above with these social network-ente eecworr-enhancedd author embeddings. Note that in the case of the weak su- pervision sctup, the tweets are generated by LLMs and there is actually no author for the tweets. As a result, in the case of the weak supervision setup, we simply average the embeddings of all augmented tweets related to perspective to get imaginary author representation for those perspectives.
We run all models (including baselines) 5 times using 5 random seeds. The average weighted Fl scores for the classification tasks for all models with the corresponding standard deviations can be found in Table 16 and the average macro Fl scores are reported Table 4.
We concatenate the shared RoBERTa-tapt en- codings of the tweets and entities with the corre- sponding learned author embeddings and use these concatenated representations as inputs for the task-
D.6 Model Implementation Libraries
We use the DGL Graph Library fhttps://www/ dgl.ai/) and Py to implement all of the mod-

---

els. We use AdamW optimizer for all of the models
annotated test and train sets in Table 3). We use a random seed of 1000 for initializing all models and the same hyperparameters as described in Sections D.3 and D.4. The following qualitative evaluations are done using the inferred labels.
for optimizing parameters.
D.7 Infrastructure
We run all of the experiments on 4 core Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz machine with 64GB RAM and two NVIDIA GeForce GTX 1080 Ti HIGB GDDR5X GPUs. GPT-J-6B was mounted using two GPUs To run our graph-based model for 300 epochs it takes around 8 hours in this in- frastructure.
E.1 Correlation of author behavior with stance
To do the analysis on the correlation of authors' following behavior with their stance on #BLM, we first compile list of tweets accounts of US politi- cians from the publicly available congressional tweets corpus available on https: thub. com/ alexli @elelitee/conggen ttl/ieegresosseeeeeeeeeeeeeeeeee the follow relationships between these US politi- cians and the authors in our dataset. Then we mea- sure the Point-biserin correlation coefficient (Tate, 1954) between the percentage of time user follows politician from specific political party (Repub- lican/Democratic) and the identified stance of the author by the models. We find all correlations with p < 0.0001. 1, 528 authors are found to follow at least one politician in our corpus.
We submit our model implementation scripts with this manuscript. All of our pretrained models and scripts for implementation will be made pub- licly available upon acceptance of this paper. A randomly generated subset of the LLM-generated training set and annotatdd test set is sub- mitted with this manuscript for review. We will also publish these datasets by maintaining proper Twitter privacy protocols upon acceptance.
Then we collect
D.8 Ablation
To determine how many training examples are needed learn our proposed self-learning ap- proach, we ran an ablation study. We randomly sample 33.33%, 66.67% and 100% of the training data from the LLM-ggnerated training set (weak supervision) and human-amann annotated training set (di- rect supervision) (as summarized in Table 3) and train our proposed model and the multitask base- line (with author embeddings) using the sampled set. We run this sampling and train the models using five random seeds (1000, 2000, 3000, 4000, 5000) and create learning curves using the average over all runs. The learning curves for combined perspective and tweet stance detection are shown in Figure 4. It can be observed that in both weak and direct supervision setups, our model is less sensitive to the number of training data compared the multitask baseline in both of the tasks. It proves that our proposed model can achieve good performance with a little amount of annotated or artificially generated data.
To analyze the correlation of authors' sharing behavior with their stance, we collect the list of left and right news media domains from https: /
com/ Then we measure the Point-biserin correlation coefficient between the percentage of time a user shares an article from media outlet with a specific bias (left/right) and the identified stance of the author by the models. We find all correlations with < 0.0001. 2, 217 authors are found to share at least one news article from the news sources we gathered. Note that, during preprocessing, all URLs are removed from the tweet texts. Hence, the media outlet domain information is not encoded in the input embeddings in our model.
The numeric values in the correlations bar plot in Figure can be found in Table 17.
E.2 Entity mapping analysis
First, we manually detect groups of literal entities that are the same. For example. "lives", "their lives" are merged into the literal entity "lives". Then we detect the high PMI perspectives associated with each of these literal entities in each of the camps ing the equation described in Appendix A. The high PMI perspectives are reported in Table 6. Some example tweets corresponding the perspectives in Table 6 and the literal entity groups are shown in Tables 20 and 21, respectively. We find that our
E Qualitative Evaluation Details
We infer perspectives using model and the mul- titask baseline on the whole #BLM corpus (in Table 1). In the weak supervision setup, we use all the tweets generated by GPT-3 as training data, and in the direct supervision setup, we use all the human- annotated data for training (combining the human-

---

Keyword- Based
Multitask (weak sup.)
Multitask (direct sup.)
Our Model (weak sup.)
Our Model (direct sup.)
Cur(Pro-BlackLMM Follow Demucrats)
0.22
0.42
0.50
0.57
0.57
Corr(Pro-BueLM)) Follow Republicans)
0.24
0.46
0.56
0.63
ContPro-BlackLMN Share Left Media)
0.37
0.44
0.48
0,54
0.53
Curr(Pro-BueLM)) M. Share Right Media)
0.37
0.44
0,48
0.54
0.53
Table 17: Point-biserial correlation coefficient values between author stance and their following and sharing
behaviors on Twitter. All the correlations had value < 0.0001.
model is able to capture the pattern when the same phrases are used by the different movements ad- dress different entities and also the mapping and sentiments towards some common named-entities (e.g.. "Derek Chauvin") better than the multitask baseline as shown in Table 18.
F Literature Review
Understanding the discourse of the #BLM move- ment: The discourse of the #BLM movement is understudied in NLP. One of the early works (Keith et al., 2017) proposed a distantly supervised EM- based approach for Identifying the names of civil- ians in news articles killed by police. Later Nguyen and Nguyen, 2018 incorporated Deep Learning methods in this task. Both of the works study only named entities and lack the study of how entities are addressed using other words (e.g.. "Thugs". "Heroes") or simply using Pronouns. Recently, Ziems and Yang, 2021 introduced the Police Vi- olence Frame Corpus of 82k news articles cov- ering 7k police killings and they studied entity- centric framing where the frame towards the vic- tim of police violence is defined the age, gen- der, race, criminality, mental illness, and attack- ing/fleeing/unarmedd status of the victim. All of these works are done on more formal texts such as news articles and do not address how different entities are addressed and what the sentiment and perspectives expressed towards them are.
E.3 Discourse of the movements
Table 19 shows the discourse of the Pro-BlackLM and pro- Moo-BlueLM camps using perspectives and moral foundations.
To analyze the discourse of the movements we identify the high PMI perspectives associated with each campaign using the same formula as described in Appendix A.
For the identification of moral foundations in the tweets, we train a RoBERTa-based classifier nut-of-domain data. We take the Twitter moral foundation dataset proposed by Johnson and Gold- wasser, 2018. In this dataset, 2k tweets are notated for one more moral foundations. We encode the tweets using our pre-trained RoBERTa- tapt (LCLS) token of the last hidden layer) and use fully connected layer to identify the moral founda- tion labels. This is a multi-label classification task. Hence, we use BCE loss and Adam W optimizer for optimization. We stop training the model if the validation accuracy does not improve for three consecutive epochs. The moral foundation classi- fication Fl score on held-out validation was 75.19%. We infer the moral foundation labels in the #BLM corpus using this classifier.
Recently, a shared task of identifying BLM- centric events from large unstructured data sources has been proposed (Giorgi al., 2021b) and Giorgi al., 2022 introduced large tweet corpus on #BLM paving the way for more studies including ours in this area. In this paper, we unify the identi- fication of co-referenced entities, perspectives to- ward them in terms of the moral roles assigned to them, and stance prediction on # the BLM move- ment on highly noisy social media texts.
The discourse of the @ellacLLivesMattee move- ment is mostly studied in social science and com- putational social science literature. For example, one line of research studied the social dynamics, ties, and in-group commitments that influence the mobilization and formation of the movement. For example, Williamson et al., 2018 found that #BLM protests are more likely to occur in localities where
E,4 Temporal analysis
We calculate the percentage of tweets where spe- cific perspective is used given day. For one type of perspective such negative actors we cal- culate the percentage taking tweets only mention- ing negative actors in a particular stance. The tem- poral trends are shown in Figure 8.

---

Literal Entities
Most ussigned perspectives by our model (direct sup.)
Most assigned perspectives by Multitask model (direct sup.) In pro-#BlackLM
In pro-BBlackIM
In pro-&BlueI.M
In pro-ABluet.M
Black Victims
blacks pos. target
blacks neg. actor
blacks pos. larett
blacks neg. actor
blacks pos, target, police neg. actor
Derek Chauvin
police neg. actor
N/A
blacks neg. actor
racism neg. actor, police neg. actor
antifa neg. actor, BLM M nag. actor
Thugs
police neg- actor
neg. actor
David Dom
blacks pos, target
police pos. farget
blacks pos. larget
police pos. target
community pos, target, police pos. target
Lives
blacks target
blacks pos. target
police pos. target
Donald Trump
government actor
republicans pos. actor demncrats neg. actor, antifa neg. actor
republicans actor
republicans pos. actor democrats neg. activ, antifa neg. actor
whites neg. actor, racism neg. actir
They
blacks pos. target
Table 18: Examples of literal entity to abstract entity mapping. Sometimes pro- pro-BlacLLM and pro- BlueLM use the same phrase to address different entities and/or perspectives. The lexicon of the literal entities in this table can be found in Table 21.
DISCOURSE PRO--BB.CKLLVEEMANTEE
DISCOURSE PRo.-BLLUBLLVESMATTEA
High PMI Perspectives MFs in Contest
Other Pers. ln Context
High PMI Perspectives MFss in Context
Other Pers. In Context
faimess/cheuinnggg
blacks pos. target, move-
authority/subversionn police pos. target, antifa neg.
police neg. actor
thovity/suhvrrsie
ment pas. actor
police pos, actor
loyaltyherayal
actor
hoyalty/hetmyaa, fair-
blacks pas. target, racism neg. actor
bllm movement neg. Ior
authorityssbbversion, antifa seg. demacents
movement pos, actor
neveleheuting
loyaltyherayal
neg. actor
meissm/rccisss neg. actor
faimessthheating,
blucks pas. targer, whites neg. acter
antifa neg. actor
authority/subversion, bllm
koyaltyhetrayll
demoemts neg. actor
neg.
faimessthheatnng,
police neg. bolm
authority/obbeesiii koyaltyyetrayal
community pos. target, antifa neg, actor
blacks pos. target
democrats neg. actor
ment pos, actor
faimesstheating,
blacks pas. farget, racismi neg. actor
authority/ubbeesdi sion,
demoerats nep, antifa
neg, actor
suth /ubbvrssoo
community pas, target
koyalty/bttryal
neg, actor
blu movement pos. get
loyaltyhhtragal, thovity/subeerid sion
blacks pos. tarpet, bllm pos. actor
kayalryfhcrryyll thority/sabeersion
police pos. actor, sorifa neg. actor
poliice pas. targer
government neg, acter
betboritysstweersinn, myyaltyfecureg
blucks pos, target, racism neg. actor
republicans pos. actor
loyaltyfictryal, fairneskchenting
denoerats nop, actor, antifa
neg, actor
faimesskehacatnng.. fectrayll
blacks pos. target, police neg. octor
patition pos. target
Table 19: Discourse of movements explained with messaging choices and Moral Foundations (MFs), Moral Foundation care/harm was used in all of the cases by both sides. Hence, it is removed from the table. This table is created using the predictions by our model in the direct supervision setup.
more Black people have previously been killed by police, Hong and Peoples, 2021 studied how social ties influence participation in the movement and Peng et al., 2019 discovered that people who join the movement in response to a real-life event are more committed to it compared to the other types. Another line of research studied the power of social media in mobilizing the movement (Mundt et al., 2018: Freelon et al., 2018; Grill, 2021). Some works have studied what properties accelerate mo- bilization. For example, Casas and Williams, 2019 studied the power of images in mobilizing the movement and Keib et al., 2018 studied what types of tweets are retweeted more related to the move- ment. De Choudhury et al., 2016 studied temporal analysis of the #BLM movement based on social media, engagement of people from different demo- graphics, and their correlation with textual features.
One major drawback of these studies is that many of them often involve expensive human studies.
There have been several attempts to understand and analyze the narrative in online text in response to the murders of Black individuals in social sci- ence. For example, Eichstaedt et al., 2021 and Field et al., 2022 studied the emotions expressed in the online text in response to murders of Black persons. Another line of research studied the fram- ing (Stewart et al., 2017; Ince et al., 2017), rhetoric functions (Wilkins et al., 2019), participation and attention to topics (Twyman et al., 201 2017), and narra- tive agency (Yang, 2016) related to the movement. Some other works studied hashtag-based analy- sis of the #BlackLivesMatter movement (Blevins et al., 2019) and the divergence of the counter- movements (c.g., the #AltLivesMatter movement) (Gallagher et al., 2018).

---

0.3
811
sharaa0
6//2209
4//22000
65/200
67/250
09/2000
611/200
671/2000
e15/200
52x/2000
0792219
e/t/ma
20
police-pos-target community-pos-targee
police-por act-
republicans pos-actor
(a) Portrayal of entities as positive targets in pro- BlueI.M
(b) Portrayal of entities as positive actors in pro- Bluef .
over time.
over time.
0.2
0.25
4
0.1
0.05
0.05
autretal
enss/oot
61770020
ent//00
0/312003
erast
520/2000
enrana
Gar200
6ns/2n00
enwana
srat2003
srarzmo
alark-a paspaanss--o---rrae target
blm blm-pos-teee pas-poosttaee target
govt-neg-actor police-neg actor
racism-neg-acts necttor
(c) Portrayal of entities as positive targets in pro-BlackLM over time.
(d) Portrayal of entities as negative actors Bro-BlacKLM over time.
Figure 8: Temporal trends in all camps for different types of entity roles identified using our model in the direct supervision setup. Note that, mostly BL. is portrayed as positive actors pro-Blackl .M and the portrayal of other entities as positive actors is significantly lower in this camp. Hence, the trend for positive actors in pro- BlackLM is not shown in this analysis.
Literal Entities
Example tweets in
Example tweets pro- Avesaaae LivesMatter
Thanks for the body cam. It obvious to anyone, #Liberals that #Rayshardroooks was violently resisting arest. #Atlanta #Alll ivesMutter
Black American
Amest the cops that killed BBreonaa #Blackl.ives Lives- Matter
Victims
Derek Chauvin, the murderer policeman who killed George Hoyd, would not have been arrested had not N/A been for the uprising in Minnesota.
Derek Chauvin
The behaviour of the American police is absolutely revolt- ing. Rucist yobs with military grade weapons empowered by fascist presideat, Why should we automatically be ex- pected to "respect" thugs? Respect should be cumed. #BlacLLivesMatter
Blackl LivesMutter is the black version of the white trash Antifa thugs. They could care less about black lives, they te thugs and trush. #RememherDDavidDoo #Alll LivesMatter
Thugs
This isn't political, all #BlackLivesMaterr David Dorn. #AhmaudArberyy Arbery. #GeorgeFloyd EBreonnaTaylor, Stop turning this into something else for reaction.
Where you all when #DavidDon shot in the head Lives ricters and TAmiiiTemmrist ? Dis- gusting silence.
David Dom
The counless black lives that have heen lost in the U.S. due to acts of #policehrutality have been tolerated & condoned by the government enough to be considered Crimes against humanity imo. think America should be accountable lovel. #BltckLivesMatee
The Police should leave. It's worth their lives for this BS anymore. #BlueLiivsMMater
Lives
for this
Trumd should get accustomed being suroundee fencing prison guards. Aprotests #BLM #BunkerBoyTrus Brump #Bunkerbaby #TrumplsAnld- iot dTrumpls ACoward HDCProttess #ArresfTrump #Black- LivesMatter
@reulDonakdTrumg has asked for Unity from Day un- like the KemoKKKrals And @ TheDocccasts and their lead- ers are the one calling for violence, not Trump
Donald Trump
HOW ARE THESE CASES NOT MAKING HEAD-
And this is what they want reform!??! Marons cunnot win!
They
LINES!! They are being murdered and nothing is being done about it. #BlackLivvss Mates
Table 20: Examples of tweets where the Pro-BlackLM and Pro-BlueI.M the same phrases address different entities and/or perspectives. The literal entities in the example tweets are bolded and underlined. The lexicon of the literal entities in this table can be found in Table 21, "N/A" means the entity is not mentioned frequently a particular campaign.

---

Literal Entities
Lexicon
Description
# georgefloyd, george floyd, georgefloyd. # breonnataylor. brconna tayla aylor, breonna, georgellyod. george, # george- Boyd H, # rayshurdbroos,, rayshard brooks, rayshurdbrook, # ahmaudarbery, ahmaud arbery, robert fuller, samuel du- bose, sandra bland, walter scott
Black Victims
American
Names of the Black American persons who were killed.
Derek Chanvin
derek chauvin, chauvin
The police officer who George Floydd.
Thugs
thugs, these thugs, thugs
negutive term address a person group persons.
Black Police officer was killed during the George Floyd protests.
David Dom
# daviddom, david dorn
their lives, lives, the lives, our lives, life, my life, his life, her life, their life, the life. a life
Lives
explanataaa..
@ realdonaldtrump. @realdonaldrrum, realdonakdrump, trump supporters, trumip, # trump. president trump, donald trump, trump2020 they
Donald Trump
45th U.S. President who is from the Republicun Party.
They
The Pronoun they.
Table 21: Description of the literal entities studied in the qualitative evaluations.
The existing studies in computational social sci- ence narrative understanding mostly rely on hashtag lexicon-based analysis of the move- ments, hence, they fail to capture the nuances in the messaging choices and often cannot differentiate between movements and representative hashtag from one movement is "hi- jacked" (Gallagher et al., 2018) by the supporters of the counter-movement.
et al., 2016; Augenstein et al., 2016). Recently, another shared task has been proposed (Pomerleau and Rao, 2017) where the task of fake-news detec- tion is studied from a stance detection perspective. In this task, a headline and a body text are given either from the same news article or from two dif- ferent articles. The task is to determine if the stance of the body text relative to the claim made in the headline is in agreement, disagreement, discussion, or irrelevant.
when
In contrast to these existing studies in CSS and NLP on #BLM, in this paper, we propose holistic technical framework for characterizing such social movements on online media by explicitly modeling the perspectives of different camps.
Identification of stance is studied using various approaches (Cignarella et al., 2020). For example, Burfoot et al., 2011 I and Sridhar et al. (2015) stud- ied stance detection using collective classification approach. Sridhar et al., 2015 and Hasan and Ng. 201 4 studied not only stances but also the reasoning behind stances and the disagreements among them. In a related work in this line, Somasundaran and Wiebe, 2010 used sakicon-based features for detect- ing "arguing" opinions, and supervised systems us- ing sentiment and arguing opinions were developed for stance classification. Sun al., 2018 proposed neural model to learn mutual attention between the document and other linguistic factors. Thomas et al., 2006 leveraged inter-document relationships and Walker et al., 2012 leveraged the dialogic struc- ture of the debates in terms of agreement relations between speakers for stance detection. Another line of research is built on relational learning-basee based methods for stance detection where various types of contextualizing and relational properties are ex- plicitly modeled and incorporated in stance detec- tion. For example, Johnson and Goldwasser, 2016b proposed a relational learning framework incorpo- rating framing and temporal activity patterns, and
Perspective Analysis and Stance Detection: Revealing perspectives in complex and deceptive discussions is an important part of discourse analy- sis and it has been studied different settings and variations in recent studies. For example. Thomas et al., 2006 and Burfoot et al., 2011 attempted to identify stances in congressional floor-debate tran- scripts (against or in support of proposed legisla- tion). Another line of research studied stances in online debate forums where the stance (pro vs. con) of a speaker on a specific issue is predicted (So- masundaran and Wiebe, 2010; Walker et al., 2012;
Hasan and Ng, 2013, 2014; Sridhar et al., 2015;
Sun et al., 2018; Xu et al., 2018; Li et al., 2018). Mohammad et al., 2016 introduced shared task of predicting stances in microblogs such as tweets where the task is to identify the stance in tweet with respect to given target (e.g. entity, issue, and so on). Consequently, more recent works have stud- ied political stances in politically controversial is- sues (Johnson and Goldwasser, 20166 20,,b;a; Ebrahimi

---

Ebrahimi al., 2016 proposed a relational model incorporating the friendship networks of the au- thors. The main drawback of such relational learn- ing approaches is that all possible solutions to the problem are explored and the one with the best gain is returned, as a result, the inference tree becomes really large and computationally expensive for large amount of data.
trained sequence to sequence models, and Yang et al., 2020 and Ng et al., 2020 proposed pretrained transformer-based approaches for data augmenta- tion. In most of these approaches, existing text data is modified to augment new data. In con- trast to these studies, Anaby-Tavor et al., 2020 and Quteineh et al.. 2020 directly estimated the text generation process using GPT-2 (Radford et al., 2019) rather than modifying existing example to generate new data.
For the analysis of perspectives in polarized top- ies, different socio-linguistic theories have been used in the literature. For example, the Moral Foundation Theory (MFT) (Haidt and Joseph, 2004; Haidt and Graham, 2007) and framing anal- ysis (Entman, 1993; Chong and Druckman, 2007; Boydstun et al., 2014). Framing is referred to the approach of communication by focusing cer- tain aspects of a story in order bias the read- ers toward certain stances. Previous studies used framing to understand political perspectives and communication strategies by biased news sources and social media users (T'uu et al., 2015; Baumer et al., 2015; Card et al., 2015; Field et al., 2018; Demszky et al., 2019; Fan et al., 2019; Roy and Goldwasser, 2020). The Moral Foundation Theory is also widely used for analyzing perspectives (De- Fulgoni al., 2016; Brady et al., 2017; Hoover et al., 2020; Roy and Gold- wasser, 2021). However, Moral Foundations are widely used to understand sentence level perspec- tives. Roy et al. (2021) introduced Morality Frames which is knowledge representation framework for capturing entity-centric moral sentiments, Because of the expressivity of entity-centric moral founda- tions and their strong correlation with stances, in this paper, we use morality frames for modeling perspectives the #BlackLivesMatter and #Bluc- Lives movements.
With the recent advances of transformer- based pretrained auto-regressive Large Language Mod- els (LLMs) such as GPT-3 (Brown et al., 2020), a new direction for data augmentation has opened up where controlled text generation (lyyer et al., 2018; Kumar et al., 2020; Liu et al., 2020) is fea- sible by prompting these pretrained LLMs. These LLMs are trained on huge corpus of web crawls, hence, have the capability to generate human-likk text. One recent work by Liu et al., 2022 combined the generative power these LLMs and the evalua- tion power of humans and proposed a human-LLLM interaction loop to generate datasets for Natural Language Inference (NLI) tasks. Inspired by these recent advances in data augmentation using LLMs, in this paper, we propose augment few-shot train- ing data using LLMs by prompting them in struc- tured way. This augmented dataset is later used bootstrap our proposed model.
hghani et al., 2014;
Data Augmentation Approaches in NLP: Data augmentation refers to the technique where addi- tional and ideally diverse data are generated with- out explicitly collecting new data. Data augmen- tation techniques can be useful in a low-resource setting where obtaining dataset for training models are difficult. Annotated data for nuanced perspec- tives are difficult to get as annotating them pertains to specialized knowledge. Hence, automatic data augmentation is desirable.
With the recent advances in deep learning-basee models, different data augmentation techniques have been proposed. For example, Kobayashi, 2018 proposed RNN-based, Kumar et al., 2019


Reviews:
[{"review_id": "uZ8OkjjeDD", "reviewer": "Reviewer_Yvsx", "summary": "", "questions": "The self-learning explanation can be clearer - for instance, it is not clear if the number of steps and epochs refer to the same k value. Could this be made clearer in the paper?\n\nThe tweet consistency again exposes the flaws that people will always be either completely pro or against movements, there is no indication of how neutral events were handled. Was there a consideration to look at the data and consider what to do with tweets/accounts which potentially had neither positive nor negative sentiment towards the event?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review!\n\n> The paper suffers from putting people into exactly two buckets - either pro or against and makes a lot of extrapolations and conclusions based on a black or white interpretation of people's views.\n\n> The tweet consistency again exposes the flaws that people will always be either completely pro or against movements, there is no indication of how neutral events were handled. Was there a consideration to look at the data and consider what to do with tweets/accounts which potentially had neither positive nor negative sentiment towards the event?\n\nIn this paper, we analyzed two extremely polarized movements namely #Blacklivesmatter and #Bluelivesmatter. It seems unlikely, counter-intuitive, and probably outlier cases when people use the phrases \"Black lives matter\" or \"Blue lives matter\" and remain neutral. In Computational Social Science studies also, mention of these two phrases is widely considered as endorsement/support or interaction/participation to the corresponding movements (please refer to the papers we cited in the literature review (L2053-L2105)), if not \"hijacked\" (Gallagher et al., 2018). This is also well-supported by our experiments where the classification of tweet stances based on these keyword mentions yields a pretty high F1 score of 87.77% (Table 4, L460-L467) in human-annotated test data. We want to reemphasize that the source dataset (Giorgi et al., 2022) we used for this study collected $\\textbf{only}$ the tweets that mention these stance indicator phrases, hence, they already have guardrails for neutral cases if there are any. Moreover, we took into account the users who tweeted at least 5 times after the killing of George Floyd (L118-L120). It ensures that only high-vocal supporters of the movements are analyzed in this paper. \n\nIn real life, there might be neutral users who do not participate in such movements, however, in this paper, our goal is to study the movements (not the neutral cases) and the perspectives that appear inside the movements.\n\nWe did not get any mention of such neutral cases from the human annotators during the in-house human annotation process as well. Also, all of our qualitative evaluations are reported at an aggregate level and quantitative evaluations are done using human annotated test set. As a result, even if the slightest amount of neutral-stance outliers are present in the data after all the guardrails, they should not affect the trends we observed during our evaluation. Hence, we believe, we did not extrapolate anything, rather honestly reported just what we observed.\n\nThere may be other movements where pro, anti, or neutral stances are almost equally common, and extending our model to those cases should be straightforward. In that case, we can just account for one additional stance that is \"neutral\" during modeling. Based on your review, we can make these statements clearer in our writing.\n\n>A number of conclusions are drawn from the analysis on this dataset - however, the authors themselves mention that the dataset is skewed and leans heavily towards pro BlackLM ...\n\nWe believe the distribution of the #Blacklivesmatter and #Bluelivesmatter tweets in the source dataset (Giorgi et al., 2022) represents real life. It has been well supported by many previous studies and survey reports that #Blacklivesmatter is a much bigger movement and its activists are way more active than #Bluelivesmatter on Twitter. For example, the sharp skewness towards the #Blacklivesmatter tweets is reported by two reputed research institutes as follows.\n\n[Article from Pew research](https://www.pewresearch.org/internet/2018/07/11/an-analysis-of-blacklivesmatter-and-other-twitter-hashtags-related-to-political-or-social-issues/)\n- \"From July 18, 2016 through May 1, 2018, the #BlackLivesMatter hashtag has been used an average of 15,856 times daily. The #BlueLivesMatter hashtag has been used an average of 3,998 times daily”.\n\n[Article from Brookings.edu](https://www.brookings.edu/articles/how-george-floyd-changed-the-online-conversation-around-black-lives-matter/)\n- “the #BlackLivesMatter hashtag has emerged as an enduring feature of online discourse. As of April 30, 2021, it has been used in more than 25 million original Twitter posts, which collectively have garnered approximately 444 billion likes, retweets, comments, or quotes—roughly 17,000 engagements per post.”\n- “Between 2013 and 2021, #BlueLivesMatter has registered 1.6 million original posts and 1.7 billion engagements”; \n\n>Some of the assumptions such as all pro blackLM follow democrats or otherwise does not hold much weight when the dataset is skewed.\n\nThe point-biserial correlation (L1937-L1970) we used to obtain the correlation between an author’s stance label and their following and sharing behavior, $\\textbf{does not}$ depend on the distribution of the dichotomous variable which is the stance of the authors in our case (either pro-BlackLM or pro-BlueLM), which is under discuss for its skewness. The continuous variables (percentage of following of Democrats/Republicans and percentage of sharing from Left/Right media) are normally distributed in our case inside each dichotomous class verified by a Shapiro-Wilk test (although strict normal distribution is not required for point-biserial). [Reference linked.](https://statistics.laerd.com/spss-tutorials/point-biserial-correlation-using-spss-statistics.php) \n\nWe found all the correlations with high statistical significance (L1937-L1970), hence, we believe they are sound. \n\nSome other qualitative evaluations are done using pointwise mutual information scores, which also take care of class distributions in the formula (Appendix E.2, E.3). \n\nAlso, in our models, we used weighted loss functions and we did a fine-grained evaluation with a human-annotated test set and we presented macro (Table 4), weighted (Table 16), and per class (Table 5) F1 scores for perspectives identification to account for the skewness in any of the classes. \n\nHence, we do not believe the skewness in the data makes any of our analyses of less weight. \n\n\n>there is no mention of exactly how this data of ~500 tweets was split into pro blm/bluelm?\n\nThe LLMs were prompted to generate tweets having either pro-BlackLM or pro-BlueLM stances along with the perspectives (Figures 5 and 6) and this method is clearly described in the paper (Section 4: L346-L361, Appendix B). Hence, the generated tweets are readily split into pro blm/bluelm labels after prompting.\n\n>However, all evaluations are done using seq2seq models even though LLM inference has been used and could potentially act as a very good baseline for classifying tweets.\n\nWe are not sure what is meant by \"all evaluations are done using seq2seq models\". We used RoBERTa-based (an encoder-only model) end-to-end and multitask models as baselines and human-annotated test data for evaluating the classification tasks. We used LLMs only for a few training data generations. Note that generative LLMs by design generate sequences, not class probabilities. Adapting them for classification tasks requires additional effort and extensive prompt engineering, particularly in problems like the one studied in this paper where more context beyond the text is needed and multiple classification decisions are needed to be made jointly. To keep our contribution focused and within the space limit, we leave it as a future work.\n\n>it is not clear if the number of steps and epochs refer to the same k value. Could this be made clearer in the paper?\n\n\nYes, step and epoch refer to the same thing. We will make it clear.\n\n> A large portion of the paper is heavily targeted at an audience which is familiar with the American political and news ecosystem. As a result, a large number of world readers might not be completely familiar with the topic.\n\nThe #BLM protests after the killing of George Floyd in the US gained much attention and spread internationally. The source dataset contains geo-tagged tweets related to #Blacklivesmatter and counter-movements from all over the world and in different languages (please refer to Georgi et al., 2022). We only focused on tweets written in English (not limited to the U.S. only) to keep our study focused (L118). However, we think our model can be applicable to any national or international movement. Hence, we do not think the movements studied and the models proposed in this paper are of limited interest.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This work presents a study on a less-studied dataset. \n2. The authors make use of Graph-neural networks to create a graph using texts to analyze social movements on social media websites which is unique. \n3. The paper is well-written and presents results clearly.", "reasons_to_reject": "1. The paper suffers from putting people into exactly two buckets - either pro or against and makes a lot of extrapolations and conclusions based on a black or white interpretation of people's views. This is a really big weakness as the authors could have addressed this by either filtering neutral tweets or adding additional guardrails and verification to throw out non extreme accounts. \n2. A number of conclusions are drawn from the analysis on this dataset - however, the authors themselves mention that the dataset is skewed and leans heavily towards pro BlackLM. Some of the assumptions such as all pro blackLM follow democrats or otherwise does not hold much weight when the dataset is skewed.\n3. A large portion of the paper is heavily targeted at an audience which is familiar with the American political and news ecosystem. As a result, a large number of world readers might not be completely familiar with the topic.", "questions_to_authors": "The self-learning explanation can be clearer - for instance, it is not clear if the number of steps and epochs refer to the same k value. Could this be made clearer in the paper?\n\nThe tweet consistency again exposes the flaws that people will always be either completely pro or against movements, there is no indication of how neutral events were handled. Was there a consideration to look at the data and consider what to do with tweets/accounts which potentially had neither positive nor negative sentiment towards the event?", "paper_topic_and_main_contributions": "This work proposes a weakly supervised graph based approach to determine perspectives during the two movements.  LLMs are used to generate training data. The authors suggest a structured representation  for analyzing online content. \n\nThe authors note that disambiguating stances towards a movement is important and they used graph-based representations for it. To account for the annotation costs, they use LLMs to generate synthetic tweet like data. The authors considered tweets posted in the month immediately following a triggering event. They use morality frames to determine the sentiment towards an entity\n\nThe authors argue that the stance of an author, the entity disambiguation and entity role are interdependent. The authors use a 2-layer R-GCN for the graph representation. The authors then propose self-learning in order to allow their models to learn constantly. \n\nThe training data is generated by prompting LLMs. However, there is no mention of exactly how this data of ~500 tweets was split into pro blm/bluelm? The authors also then go on to perform the experiments with their modeling setups. They find that using their graph techniques and a self-learning framework yields superioir results to a baseline model. However, all evaluations are done using seq2seq models even though LLM inference has been used and could potentially act as a very good baseline for classifying tweets. \n\nThe authors also note that the dataset is heavily skewed. This makes one question some of the analysis as making extrapolations based on skewed datasets might not hold.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8nJtDBEuXz", "reviewer": "Reviewer_GbZ4", "summary": "", "questions": "A. How generalizable is your method - ie, how much work would a new researcher need to do (eg, in defining entities) to apply this method to another social phenomena with multiple perspectives?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review!\n\n> Weak supervision could be better evaluated - eg, how realistic are the evaluated tweets? The prompt requires \"all of the structured elements for perspectives to be present in the generated tweets\", which doesn't see the most realistic. The generation of authors is also not realistic (\"[author] embeddings are initialized by averaging the corresponding artificial tweets\").\n\nWe found the generated tweets very realistic. Please refer to Table 11 for generated tweet examples. Also, a sample of the tweets is submitted with this manuscript. The structured elements required in the prompts are well defined, hence, we believe it is realistic. The dummy author nodes were generated only in the setting when the tweets were augmented using LLM; when a real author exists, this step is not required in our model.\n\n> The authors also claim that weak supervision achieves \"comparable\" performance - this feels like a bit of an overstatement. For author stance, performance drops 5 points from direct to weak (same gap as from their model to baseline); also substantial drops for ambiguous (30 points) and entity mapping (8 points).\n\nIn the other tasks, weak supervision achieves pretty comparable performance to direct supervision and it outperforms most direct supervision baselines. In the immediate next sentences, we discussed the reasons for the failure of weak supervision in the addressed tasks (L489-L494). Regardless, we will rephrase this statement. \n\n> A. How generalizable is your method - ie, how much work would a new researcher need to do (eg, in defining entities) to apply this method to another social phenomena with multiple perspectives?\n\nWe believe our method is generalizable to any social phenomenon. The new researchers just have to put a one-time effort into identifying the main actors (defined as abstract entities in our paper) related to the phenomena if the actors are not already given. As discussed in our limitations section (L666-L672), the main actors related to a movement can be extracted from an external database, or by doing a one-time human-in-the-loop data analysis as we did in this paper. The other properties of perspectives such as pro/anti-stances, moral sentiments, etc. are already well defined in the literature.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The authors introduce an innovative methodology that jointly performs a number of tasks related to perspective identification. While the graph neural network is standard (R-GCN), it is innovative to define several prediction tasks from the same node embeddings and to capture their dependencies by training them jointly and enforcing consistency.\n- Their model achieves strong performance and outperforms baselines, and many aspects of their method (representing tweets as graph, adding the author relations to the graph, adding self-learning) improve the model performance.\n- The task is challenging and interesting - identifying differing stances on the same topics - and shows up frequently in analyses of social data and social phenomena.", "reasons_to_reject": "- Weak supervision could be better evaluated - eg, how realistic are the evaluated tweets? The prompt requires \"all of the structured elements for perspectives to be present in the generated tweets\", which doesn't see the most realistic. The generation of authors is also not realistic (\"[author] embeddings are initialized by averaging the corresponding artificial tweets\").\n- The authors also claim that weak supervision achieves \"comparable\" performance - this feels like a bit of an overstatement. For author stance, performance drops 5 points from direct to weak (same gap as from their model to baseline); also substantial drops for ambiguous (30 points) and entity mapping (8 points).", "questions_to_authors": "A. How generalizable is your method - ie, how much work would a new researcher need to do (eg, in defining entities) to apply this method to another social phenomena with multiple perspectives?", "paper_topic_and_main_contributions": "This paper develops a methodology for identifying perspectives in tweets related to the Black Lives Matter movement. They focus on two stances: pro-BlackLivesMatter (pro-BlackLM) and pro-BlueLivesMatter (pro-BlueLM). They define perspectives in terms of sentiment (positive/negative) toward entities (eg, Black Americans, Police) and their roles (actor/target).\n\nTo identify perspectives, they develop a graph-based methodology that unifies a number of prediction tasks. First, they construct a graph representing the tweets (with tweet-entity and tweet-keyword relations) and authors (with author-tweet and author-author relations). They use relational GCN (for multiple relation types) to learn representations of the nodes. Then they define a series of prediction tasks--tweet stance prediction, entity sentiment and role prediction, and entity stance prediction--that are all predicted from the node embeddings, thus unifying the tasks under a common framework. Finally, they incorporate self-learning, where the model begins with a small amount of annotated data, then learns to add training data based on consistency (eg, a tweet stance that is pro-BLM is consistent with the perspective police-neg-actor).\n\nTo train the models, they try direct supervision and weak supervision. For direct supervision, they sample 200 users and manually annotate their tweets. For weak supervision, they generate tweets using GPT-3. They try training with real train data and generated data, and evaluate on real test data in both cases. They find that their model outperforms baselines and that each aspect of their method (representing tweets as graph, adding the author relations to the graph, adding self-learning) improves the model's performance. Direct supervision outperforms weak supervision, but sometimes not by much. Finally, they conduct some qualitative analyses and find expected correlations, eg, between stance and political affiliation, and they find their correlations are stronger for their model than for baselines.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ErgW8A3svG", "reviewer": "Reviewer_qRYV", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! \n\n$\\textbf{GPT-3 usage:}$ Usage of GPT-3 is neither the central theme of our paper, nor our model solely depends on GPT-3. The main focus of this work is to disentangle the discourses on the #Blacklivesmatter and #Bluelivesmatter movements by explicitly modeling the perspectives in them. Our proposed model requires only a few training data samples to identify perspectives in a large social media corpus. This little amount of training data can be – artificially crafted or real data annotated by humans. To support this claim we performed all our experiments separately using GPT-3 generated and human-annotated training sets and tested only on human-annotated test data. Even if we drop the results that we obtained using the GPT-3 generated training data, the main claims and findings of our paper do not change at all. Also, we do not believe, the usage of GPT-3 for such a non-central and small aspect of our model makes our whole modeling approach non-reproducible. \n\nWe prompted GPT-3 to produce some coherent text given a stance, entity, hashtag, and moral sentiment towards the entity. Hence, we are not extracting any additional information from the LLM. Moreover, one author of this paper manually went through the generated few-shot examples to observe if any additional information was added to the tweets. Please refer to section 4, Appendix B for the method and Table 11 for generated examples. \n\nThe capability of GPT-3 in various tasks is being explored widely in recent scientific works and they are being accepted in highly reputed scientific publication venues (e.g., ACL, NeurIPS, etc.). Hence, we believe the way we used GPT-3 in this paper is not at all non-scientific. We are listing a couple of recent papers that used GPT-3 and were published in ACL and NeurIPS below for reference.\n\n- “When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment.” Jin et al. NeurIPS 2022\n\n- “Does GPT-3 Grasp Metaphors? Identifying Metaphor Mappings with Generative Language Models”. Wachowiak and Gromann. ACL 2023.\n\n\n\n>The underlying assumptions of the self-learning loop are fairly strong: that a tweet must not express ambiguous / ambivalent stances about the issue and that an author never changes their mind. \n\nWe are dealing with highly polarized movements and the source dataset we used contains only the tweets that mentioned support-indicating keywords (#Blacklivesmatter, #Bluelivesmatter), not neutral cases. To infer stances and perspectives in such polarized movements, we deployed the self learning loop based on consistency in authors' stance and perspective. However, the self-learning loop does not prevent us from finding ambiguous and interesting cases. For example, when the supporters of a specific movement use the keyword of the opposite movement to \"hijack\" the discussion (supported by previous work by Gallagher et al., 2018) (L391-L397). This is studied in depth in our paper and our finding is that in such ambiguous cases also, our model excels (Table 4, L460-L494). Because, even when the keywords/hashtags are hijacked, the author’s social media interaction and perspective toward the main actors, help disambiguate such deceptive tweets, indicating the effectiveness of the assumptions in the self-learning loop. \n\nWe studied the discourse of the #BlackLivesMatter and #BlueLivesMatter movements in the timeframe of 1 month after George Floyd's killing. It is highly unlikely that an author’s stance will change in such a short period. We acknowledged such assumptions in the Limitations section and will make it clearer.\n\n> it doesn't really produce any insight in the end that couldn't be quickly obtained by human annotators inspecting a sample of tweets on both sides\n\nAs stated above and thoroughly studied in the paper, our main focus is to disentangle opposing movements by explicitly modeling and inferring perspectives in a large social media corpus (400k tweets) by using only a few training examples (~500 tweets). We do not think, identifying perspectives in such a large corpus is feasible with human annotation as it involves difficulties in entity disambiguation and stance inference that requires more context than just text and specialized human knowledge. This claim is explored in depth by the experiments presented in the paper. \n\n>What remains is the insight that text-as-graph and social features improve the quality of such models. Ok, but this is already well known in the field on a variety of use-cases\n\nTo the best of our knowledge, none of the prior works (including the ones you mentioned) studied text-as-graph in the same spirit as we did, and none combined text-as-graph and the social media interaction graph. However, we took inspiration from prior influential works that used social network information in text understanding (however, mostly studied text as a single unit) in NLP and are adequately cited in our paper (L616-L624). \n\n>Fake user detection on Twitter: Balaanand, Muthu, et al. \"An enhanced graph-based semi-supervised learning algorithm to detect fake users on Twitter.\" The Journal of Supercomputing 75 (2019): 6085-6105.\n\nThis paper does not really study textual content in depth, rather interactions of users with tweets such as fraction of retweets, average time between tweets, and some shallow textual features like tweet length, etc. are used as features to classify fake users.\n\n\n> Sentiment analysis on Twitter: Lovera, Fernando Andres, Yudith Coromoto Cardinale, and Masun Nabhan Homsi. \"Sentiment Analysis in Twitter Based on Knowledge Graph and Deep Learning Classification.\" Electronics 10.22 (2021): 2739.\n\nThe authors in this paper extracted the dependency graph (PoS tags) in text by using a dependency parser (and addressed it as knowledge graph) and used that for sentiment analysis which is absolutely not related to what we mean by text-as-graph in our paper. Our focus is on a rich semantic graph representation for text consisting of entities, moral sentiment towards entities, stance indicator hashtags, and connecting that with the social network interaction of the authors for getting a rich contextualized representation of text (Section 3.1). The above paper did not include any social interaction, and the authors did not even compare their model with other state-of-the-art NLP models at that time. Most concerningly, it was published in a journal that was listed as predatory, and accused of self-citation, and treachery many times in the last 10 years (currently it is listed as a predatory journal in at least one ranking).\n\n>Detecting toxicity on Twitter: Jiang, Jian, Rui Wang, and Guo-Wei Wei. \"GGL-Tox: geometric graph learning for toxicity prediction.\" Journal of chemical information and modeling 61.4 (2021): 1691-1700.\n\nWe want to respectfully indicate that this paper you suggested is not a study on toxicity on Twitter, rather it is an article related to drug design and discovery, and the authors apparently studied toxicity in environmental chemicals and drugs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This work focus on a very important contemporary topic: the understanding of opposing political movements within the context of a highly polarized society and mass communication through social media. The characterization of perspectives is grounded on Morality Frames theory from the field of linguistics and translates it successfully to a deep learning based NLP framework.\n\nThe deep learning model is technically sound, the choices are well justified and the experiments are performed in a rigorous fashion.", "reasons_to_reject": "Unfortunately, I have one major objection to one aspect of the methodology, that I believe makes this work unsuitable for publication in a scientific conference: the use of a proprietary, black-box LLM or the generation of synthetic training cases. It is completely unacceptable to rely on proprietary black-boxes for scientific research, and if we allow for this we risk doing irreparable damage to the field. We can't really know what the model behind the API is, nor can we know how it was trained. It seems fairly obvious that GPT-3 training corpus included tweets, but we cannot know which ones, selected according to what criterion. This \"weakly supervised\" approach is in fact extracting knowledge from a black box. I have no doubt that this worked well in this case, as the authors report, but this is useless in a scientific setting.\n\nI imagine that the aspect of the article that relies on GPT-3 could be extracted, but even then I believe that big issues remain. The underlying assumptions of the self-learning loop are fairly strong: that a tweet must not express ambiguous / ambivalent stances about the issue and that an author never changes their mind. Such assumptions clearly work well in producing consistency, and it looks that it works well for this specific case, but it doesn't really produce any insight in the end that couldn't be quickly obtained by human annotators inspecting a sample of tweets on both sides. Could it be that introducing such strong consistency requirements throws the baby with the bath water? Won't it guarantee that nothing interesting is actually detected? After all, the point of automating text understanding is to produce insights that could not be obtained by traditional methods with a small amount of effort.\n\nWhat remains is the insight that text-as-graph and social features improve the quality of such models. Ok, but this is already well known in the field on a variety of use-cases, for example:\n\nFake user detection on Twitter:\nBalaanand, Muthu, et al. \"An enhanced graph-based semi-supervised learning algorithm to detect fake users on Twitter.\" The Journal of Supercomputing 75 (2019): 6085-6105.\n\nSentiment analysis on Twitter:\nLovera, Fernando Andres, Yudith Coromoto Cardinale, and Masun Nabhan Homsi. \"Sentiment Analysis in Twitter Based on Knowledge Graph and Deep Learning Classification.\" Electronics 10.22 (2021): 2739.\n\nDetecting toxicity on Twitter:\nJiang, Jian, Rui Wang, and Guo-Wei Wei. \"GGL-Tox: geometric graph learning for toxicity prediction.\" Journal of chemical information and modeling 61.4 (2021): 1691-1700.", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this work the authors propose a graph-based learning approach to jointly learning socio-semantic representations of the opposing \"Black Lives Matter\" and \"Blue Lives Matter\" movements as expressed in a corpus of tweets. The authors define perspectives by adapting Morality Frames theory and capturing two dichotomies about identified abstract entities: actor / target (having agency or being acted upon) and sentiment (positive / negative). A multi-relational graph is considered, combining relations between author and tweet, author and author, author and keyword, hashtag and tweet and entity and tweet.\n\nA self-learning loop is employed, where the training set is iteratively augmented with tweets and entities that are deemed to be \"consistent\". Shortly, a tweet is consistent if its predicted stane is aligned with the predicted perspectives of the entities mentioned in the tweet and an author is deemed consistent if all of their consistent tweets have the same alignment. Furthermore, the authors propose a \"weakly supervised\" approach where synthetic training data is generated with GPT-3, as a strategy to mitigate the effort inherent in annotating large text corpora. In all cases, models are tested with human-labelled data.\n\nF1 scores show that the framework proposed outperforms multitask RoBERTa and ablation tests show that the representation of text as graphs of entities, the use of the author network and the self-learning loop all contribute to higher scores. Common perspectives on various entities from both Black Lives Matter and Blue Lives Matter are presented, and they agree with one's intuitive understanding of theses movements.", "excitement": 2, "reproducibility": 1, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "Black-box proprietary models that one can only access through an API should not be accepted as part of valid scientific methodology and such behavior should not be normalized."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Thank you to the authors for their work and the reviewers for their reviews. This work focuses on automatically generating perspectives about abstract entities in polarized social media text, focusing on #BlackLivesMatter and #BlueLivesMatter tweets.

Pros:
- All reviewers rate the paper high for soundness
- As noted by GbZ4 "The task is challenging and interesting - identifying differing stances on the same topics - and shows up frequently in analyses of social data and social phenomena." This methodology has the potential to be broadly useful for social text analysis
- GbZ4 and Yvsx agree that the graph-based methodology with several prediction tasks is innovative

Cons:
- Several reviewers raise concerns about how the authors do not allow for neutral or ambiguous stances. As the #BlackLivesMatter movement has been highly polarized and the authors remove any users who tweet about the events <5 times, I don't see this as a fundamental flaw in this particular case study. However, the authors could add more clarity on how methods might need to be changed to use this methodology in other settings or if there are settings where they are not appropriate to use

qRYV questions the reproducibility of the work because the authors a paid API (GPT-3) for one small part of their analysis. I don't agree that using a black-box API makes the work inappropriate for academic publication, and I don't believe the authors should be penalized for this in the absence of a higher-level ACL policy precluding comparisons with APIs.

I think the task and methodology in this work will be of interest to NLP and computational social science researchers.
Decision: Accept-Findings