Detecting Propaganda Techniques in Code-Switched Social Media Text (2023)

Abstract: Propaganda is a form of communication intended to influence the opinions and the mindset of the public to promote a particular agenda. With the rise of social media, propaganda has spread rapidly, leading to the need for automatic propaganda detection systems. Most work on propaganda detection has focused on high-resource languages, such as English, and little effort has been made to detect propaganda for low-resource languages. Yet, it is common to find a mix of multiple languages in social media communication, a phenomenon known as code-switching. Code-switching combines different languages within the same text, which poses a challenge for automatic systems. Considering this premise, we propose a novel task of detecting propaganda techniques in code-switched text. To support this task, we create a corpus of 1,030 texts code-switching between English and Roman Urdu, annotated with 20 propaganda techniques at fragment-level. We perform a number of experiments contrasting different experimental setups, and we find that it is important to model the multilinguality directly rather than using translation as well as to use the right fine-tuning strategy. We plan to publicly release our code and dataset.

## introduction the rise of social media has brought a significant change in the way people consume and share information. with the advent of the internet and the widespread use of social media platforms, it has become easier for anyone to spread information to a wide audience. social media platforms in the digital era have become the primary source of news consumption and are used on a daily basis. warning: this paper contains examples and words that are offensive in nature. unfortunately, this democratic channel was also used by malicious users for harmful purposes, e.g., to spread disinformation and hate speech. an important tool to make such content believable and viral is the use of propaganda techniques. propaganda is the dissemination of biased or misleading information in order to manipulate people's beliefs and opinions towards a particular objective. propaganda has always existed, but social media platforms have made it easier for individuals and organizations to promote their agenda and narratives quickly across large audiences. propaganda and the spread of misleading claims and opinions negatively affects many people and may cause harm. this was particularly seen during the covid-19 pandemic when incorrect details about the virus and the effectiveness of vaccines spread on social media, leading to confusion and mistrust. propaganda is also used as a tool to manipulate people's emotions and behavior, often for political or ideological purposes. this can result in a decline in critical thinking abilities, impairing the capacity to make well-informed decisions grounded in factual information and evidence. thus, there is a need to detect the use of propaganda and to combat the negative impact it poses as well as to promote a more accurate and truthful exchange of information in social media. social media platforms such as facebook, twitter, youtube, and instagram are used by millions of people around the world, and many of these users are bilingual and even multilingual. due to the increase of multilingual users and informal interactions on social media, a large number of people have begun to resort to mixing multiple languages on various social media platforms in the form of posts, tweets, comments, and especially chats. this is where the term code-switching comes into play. code-switching primarily refers to switching or alternating between two or more languages in the same context @xcite . this switching may occur within the same sentence or between neighboring sentences. this phenomenon of code-switching or mixing multiple languages has now become prevalent on social media platforms among bilingual and multilingual communities. it often compensates for a lack of proper expression. this helps individuals to express their ideas and thoughts more accurately and convincingly and to achieve a better impact on the readers and the listeners @xcite @xcite . in this work, we aim to detect propaganda techniques in code-switched text: we use english and roman urdu (i.e., urdu language written using latin script) as our high-resource and low-resource languages. although urdu is typically written in its native arabic script, most online users prefer to use roman urdu in their online textual communication due to the convenience of typing. figure 1 shows an example of a code-switched text (english and roman urdu), where the different fragments of the text are labeled as propaganda techniques. significant research efforts have been dedicated to various code-switched tasks in the field of natural language processing (nlp), such as language identification, named entity recognition (ner), pos tagging, sentiment analysis, question answering, and natural language inference (nli) @xcite @xcite . however, there has been limited exploration in the domain of propaganda detection, particularly for low-resource languages. previous work in this area has focused mainly on high-resource languages such as english, with propaganda detection at the document-level as both a binary-class and a multi-class classification task @xcite , as well as at the fragment-level for journalistic news articles @xcite . our research represents the first attempt to address propaganda detection in code-switched low-resource languages on social media. however, we face several challenges in this task, including a lack of specialized corpora, highquality and reliable annotations, and fine-tuned models. our approach aims to overcome these challenges by focusing on propaganda detection on social media platforms. we can summarize the contributions of our work as follows: ## related work propaganda originated in the 17 th century in the form of agendas that were propagated during gatherings and events such as carnivals, fairs, religious festivals, and theatres @xcite . from there, propaganda took the form of text and pictures in newspapers and posters. during the american revolution in the 18 th century, newspapers and the printing press in various colonies propagated their views to promote patriotism @xcite . during the first world war, the british government organized a large-scale propaganda campaign, which was claimed by military officials to have played a key role in the defeat of the germans in 1914 @xcite . later, in the early 20 th century, after the advent of motion pictures, this medium became a new propaganda tool used to promote military and political agendas. the artificial intelligence (ai) community has primarily focused on textual propaganda. with the widespread accessibility of the internet in the past two decades, propaganda has proliferated at an enormous scale, compelling research communities to devote efforts towards detecting propaganda in its various forms. initially, research on propaganda in ai started at the document level. @xcite performed binary-class propaganda detection at the news article level. their model assessed the level of propaganda based on the presence of keywords, specific writing style, and readability level. @xcite focused on distinguishing between four types of documents in their annotated corpus (tshp-17): satire, hoax, trusted, and propaganda. they took an analytical approach to the language of the news media in the context of political fact-checking and fake news detection. @xcite took a more finegrained approach to propaganda detection. to counter the lack of interpretability caused by binary labeling of the entire article, they proposed to detect propaganda techniques in spans/fragments of the text. the annotation in their corpus was at the fragment level, and featured 18 propaganda techniques. this allows for two tasks: (i) binary classification (whether propaganda is present in a sentence or not), and (ii) multi-label classification (where a span of text in a sentence can be labeled as one or more classes from a total of 18 techniques). the list of the 18 propaganda techniques was selected due to their occurrence in journalistic articles. this task led to the creation of the ptc corpus @xcite , which contains 451 news articles from 48 news outlets. there were also some related shared tasks. semeval-2020 task 11 @xcite focused on detecting persuasion techniques in news articles. the task involved identifying specific text spans and predicting their corresponding types (14 different techniques). @xcite extended the scope of fragment-level propaganda identification in semeval-2021 task 6: the task includes the detection of propaganda techniques present in both textual and image data. in a wanlp'2022 shared task, @xcite focused on detecting propaganda techniques in arabic, specifically using arabic tweets. a follow up wanlp'2023 task, focused on propaganda in tweets and news articles @xcite . moreover, @xcite (semeval-2023 task 3) worked on detecting the category, the framing, and the persuasion techniques in a multilingual setup. this work makes use of news articles in nine different languages during training and evaluates on additional three languages in the testing phase. @xcite focused on multimodal propaganda detection, as a multi-label multimodal task that focuses on propaganda detection in memes. they released a corpus of 950 memes, labeled with 22 propaganda techniques. @xcite developed an interpretable model for propaganda detection. more recently, baleato rodríguez et al. (2023) performed multi-task learning with propaganda identification as the main task and metaphor detection as an auxiliary task. their work makes use of the close relationship between propaganda and metaphors. they leveraged metaphors which in turn improves the model performance, especially while identifying name calling and loaded language. @xcite presented a new multilingual multi-facet dataset comprising 1,612 news articles used for understanding the news (particularly "fake news"). the dataset includes annotated documents categorized by genre, framing, and rhetoric (persuasion). the annotation of persuasion techniques is conducted at the span level, using a taxonomy that consists of 23 techniques organized into six categories. ## results through various experiments, our study has revealed that training models on code-switched text yields superior performance for our propaganda detection task compared to training on translated versions of the code-switched text in high-resource languages. these findings emphasize the significance of fine-tuning the models specifically on code-switched text, as opposed to relying on the translated versions of the text in high-resource languages. the potential reason for the lower performance that we observed in translating code-switched data to high-resource languages could be attributed to the translation service's inability to capture and preserve the inherent propagandistic characteristics present in code-switched text from low-resource languages. we also observe that the cross-lingual model's ability to transfer knowledge and capabilities from one language to another one allows it to exhibit superior performance compared to both monolingual and multilingual models when handling code-switched text. ## conclusion and future work we proposed the novel task of detecting propaganda techniques in code-switched social media test. we created a corpus of 1,030 code-switched english and roman urdu texts annotated with the use of 20 propaganda techniques. we performed a number of experiments using different fine-tuning strategies and we found that modeling the multilinguality directly rather than using translation of the code-switched text yields better results. in future work, we plan experiments with detecting propaganda techniques in code-switching text for other resource-poor languages. we further plan to expand our annotated corpus with many more examples. we also want to fine-tune pre-trained multilingual and cross-lingual language models specifically aiming at detecting fine-grained propaganda at the fragment level. last but not least, we want to understand the best strategy for handling codeswitched text with llms.