TWEETSPIN : Fine-grained Propaganda Detection in Social Media Using Multi-View Representations (2022)

Abstract: Recently, several studies on propaganda detection have involved document and fragment-level analyses of news articles. However, there are significant data and modeling challenges dealing with fine-grained detection of propaganda on social media. In this work, we present TWEETSPIN, a dataset containing tweets that are weakly annotated with different fine-grained propaganda techniques, and propose a neural approach to detect and categorize propaganda tweets across those fine-grained categories. These categories include specific rhetorical and psychological techniques, ranging from leveraging emotions to using logical fallacies. Our model relies on multi-view representations of the input tweet data to (a) extract different aspects of the input text including the context, entities, their relationships, and external knowledge; (b) model their mutual interplay; and (c) effectively speed up the learning process by requiring fewer training examples. Our method allows for representation enrichment leading to better detection and categorization of propaganda on social media. We verify the effectiveness of our proposed method on TWEETSPIN and further probe how the implicit relations between the views impact the performance. Our experiments show that our model is able to outperform several benchmark methods and transfer the knowledge to relatively low-resource news domains.

## introduction propaganda refers to any idea or information, that is often false or exaggerated, and is used to promote or publicize a particular cause or point of view. in recent years, there has been a surge in research and development of methods to detect propaganda from text. for example, some of the earlier works like @xcite and @xcite released a corpus of news articles containing coarse-grained document-level annotation of propaganda. @xcite described a corpus of news articles containing annotations of 18 fine-grained propaganda techniques. following this work, two subtasks were presented as a part of nlp4if workshop @xcite that focused on the identification of propagandist text units at fragment and sentence level. we believe that the challenge for this task lies in the varied nature of propaganda techniques, including cognitive and information distortion and logical fallacies. the challenges are further exacerbated in the social media setting, which has become a key battleground in the spread of propaganda. research on propaganda detection in social media platforms like twitter has been limited by the: (a) lack of sufficiently annotated social media propaganda data, (b) idiosyncratic nature of the content on social media, (c) difficulty in modeling the social context in which propaganda is disseminated, and (d) varying propaganda techniques that require factual knowledge, structural relationships, and reasoning abilities. in this work, we address a subset of both the data and modeling challenges. first, we introduce tweetspin, a corpus of tweets containing weak labels of fine-grained propaganda techniques. we accomplish this through a data collection pipeline that incorporates keyword-based search and users calling out propaganda techniques publicly on twitter (see figure 1 for examples). next, we present a transformer-based multi-view propaganda detection model, mv-prop, that identifies varied aspects of the textual data using multi-view contextual embeddings and captures their interaction via pairwise cross-view transformers. the main contributions of this work are described as follows: i see no reason to denounce, condemn, or other ise even care about "abc" challenging the electoral college. democrats have done it since 2000 in each election here the gop on and no one cared then either. whataboutism again? "abc" is a fraud. no citi en of an part should vote for him for an thing again. that s all ou had to sa . a ne xyz poll sho s that 47% of likel voters believe "there as enough fraud to ensure abc ould in" this isn't some sort of 'conspirac theor ' hen half of america believes it. where is the activist media? where is the doj? rt! this is hat's called the band agon fallac . just because a lot of people believe something, that doesn't make it true. to be treated like a human being, and i don't ant to be infantili ed in order to have m thoughts and feelings considered b men. what about us ho's abusers ere in the famil ? those men ere alread treating us like their daughters, sisters, cousins. user 1 user 2 your part has literall spent decades falsel accusing pro-choice americans of murder. you are tr ing to justif murder. this has been done before -the nazis defined certain people as less than human before slaughtering them. reductio ad hitlerium is a technical foul. you ma as ell condemn people for earing hugo boss because the na is did. user 1 user 3 user 2 fi it. i don t atch an thing president biden sa s or does. he does not instill confidence. he makes this countr appear eak. this countr needs a leader , one ho ll take command! what has he done to make the countr look eek? what about his policies has made ou feel like ou can't rel on him. in the first 100 da s, he has done more for the people than trump did in the last si ears since he started talking about being president. trump so ed division i don t kno hat our looking at but i loved president trump. he made me and plent of people proud to be americans. it s all about perspective. i ll stand b him hether he seeks the office again or supports homever he feels is best for this countr . so, ou said president biden make ou ill, i rebutted that, and then ou use an unrelated topic to counter m rebuttal. this is called a red herring. it is a fallac . you should look at ho fallac rules our orld. user b user c user c user b user a t 2 t 3 t 4 t 5 to to propaganda k e pansio t eet filtra edge into the representation and capture their pairwise interactions through cross-view transformers. to the best of our knowledge, this is the first study that incorporates such multi-view representations for the propaganda detection task. ## related work this work is closely related to a broad spectrum of topics, including offensive language detection, computational argumentation, and fake news detection. some of the propaganda techniques might overlap with the categories of argumentation techniques (e.g., strawman argument) or offensive language (e.g., name-calling). however, there are considerable variations in propaganda techniques across different social contexts. thus, methods used for the detection of offensive language or argumentation techniques may not be directly transferable to fine-grained propaganda detection. in this section, we review the prior researches in two main areas that are relevant to our work -(a) propaganda analyses and detection and (b) multi-view representation learning. ## problem definition given an input tweet text along with a discussion context, our goal is to predict if the input text exhibits any propaganda techniques. determining the propaganda technique of the input tweet can be formulated as a multi-class classification problem. in this work, we denote the input tweet text as t @xmath0, w 2 , .., w n i ], where n i is the sequence length of the input text. each tweet text may be accompanied by a context c @xmath1, t 2 , ..., t i-1 ], referring to m prior tweets in the discussion thread related to the input tweet t i . every input text might not necessarily contain a context (in such a case, @xmath2. in this task, the target is to develop a model that can learn a mapping function f : @xmath3, where @xmath4, 2, ..., l, p @xmath5. ## tweetspin we construct a dataset of english-language tweets, referred to as the tweetspin corpus, containing weak annotations of 18 propaganda techniques same as in @xcite . table b1 shows the full statistics of our dataset. our data collection pipeline consists of three components: (a) propaganda keyword expansion, (b) keyword-based tweet retrieval, (c) tweet filtration and (d) data augmentation. propaganda keyword expansion: for each propaganda technique, we select the name of the technique as the initial keyword. this works well for techniques that cannot be easily characterized by specific lexical patterns. typical examples of such techniques include red herring, obfuscation, causal oversimplification, and strawmen. additionally, we expand the list of keywords related to some of the propaganda techniques by combining information from publicly available resources including technique-specific phrases, idioms, and examples listed in table b2 . keyword-based tweet retrieval: we use twitter's standard search api to ingest tweets based on the keywords identified from the previous step. we enclose the keywords or phrases within quotation marks. we observed that many twitter users explicitly call out any usage of specific propaganda techniques in their discussions on the platform. therefore, we select quoted tweets and replies containing keywords related to the propaganda techniques. additionally, we search for tweets containing both the word "you" along with the identified keywords to capture instances where the users call out the usage of propaganda techniques. tweet filtration: given the tweets retrieved based on the keywords, we remove tweets that are extremely short (less than 5 words) and those that are replies to tweets from deleted or protected accounts. for the remaining tweets, we collect the discussion thread for each tweet which provide the context for the tweet. though the tweet threads may involve complex tree structures with different reply branches, we are only interested in the specific branch of the tree that contains the tweet being called out for using specific propaganda techniques. in tweet threads that are long (>50 tweets in a discussion thread), not all tweets in the discussion context might be important for the classification task. thus, we apply temporal filtering on such discussion threads, where we only retain the source tweet and discussion context that falls in a 7-day window before a particular tweet was being called out as propagandistic. ## mv-prop: multi-view propaganda detection here, we describe our proposed model for finegrained propaganda detection on social media. unlike the fragment-level classification task used for propaganda detection in news articles, we formulate this problem as a text classification task where we aim to map the input tweet text to one of the several propaganda labels conditioning on the discussion thread context if it exists (as explained in section 3). figure 2 illustrates the overview of our model architecture. inspired from the literature in multi-modal learning @xcite , we propose a transformer-based multi-view propaganda detection model, mv-prop, that integrates multiview contextual embeddings via pairwise crossview transformers. the main motivation behind such a modeling choice comes from the fact that different propaganda techniques require focus on varying aspects of the data. for example, propaganda techniques like loaded language can be identified from the usage of specific words or phrases, while repetition or red herring necessitates a contextual understanding of the tweet. similarly, most propaganda techniques can benefit from word sense disambiguation and entity information that can be accumulated from external knowledge sources. thus, the fine-grained differences between these propaganda techniques call for multi-view representations that can unravel such variations in the data. our mv-prop model comprises the following components: multi-view encoding layer, which computes multi-view representation from input tweet t i and context c i . cross-view transformer, that reinforces representations obtained from a specific view with those computed from another view. we compute this for all pairs of such cross-view transformers. classification layer, which fuses the embeddings from the previous step and computes the likelihood of the input tweet with the given context belonging to a particular propaganda label. ## training & implementation details we optimize the categorical cross-entropy loss between the predicted and true propaganda labels as in equation 10 , where l is the total number of propaganda labels, q j is the predicted distribution that the input tweet falls under propaganda technique j, and p @xmath6, 1} denotes the ground-truth of whether the input tweet can be categorized under the j th propaganda label. we use the publicly released default pre-trained model parameters for the bert variants used. we perform a grid-search and optimize the hyperparameters using the validation set: n @xmath7, n @xmath8. we used adam with a learning rate of α=2e-5 and a warmup proportion of 0.1 for optimization. to account for randomness, we report the numbers which are the mean of five experimental runs with different random seeds. to alleviate the problem of unbalanced datasets, we utilize class weights in categorical cross-entropy loss based on the training and validation sets. see appendix a for details on the hardware. ## experiments our experiments are designed to investigate the following research questions: rq1: how well does our mv-prop model perform compared to the other baselines in the propaganda detection task on social media data? rq2: what are the influences of different views and their interactions on the overall performance? rq3: can our model be applied to detect propaganda on in-domain and cross-domain datasets? ## conclusion we introduced tweetspin, a corpus of tweets containing weak labels of fine-grained propaganda techniques. next, we presented a transformerbased multi-view propaganda detection model, mv-prop, that integrates multi-view contextual embeddings via pairwise cross-view transformers. we demonstrate how the semantic, relational, and knowledge view enrichment of the input tweet text leads to significant performance improvement over other baseline methods. our experiments also demonstrated the transferability of our trained model to propaganda detection for news articles. the main limitation of our work is the reliance on weak annotations of twitter data, which is unavoidable given the scale of our dataset. future work could investigate leveraging the multi-view representations for span-level detection of fine-grained propaganda techniques.