Modelling Political Aggression on Social Media Platforms (2023)

Abstract: Recent years have seen a proliferation of aggressive social media posts, often wreaking even real-world consequences for victims. Aggressive behaviour on social media is especially evident during important sociopolitical events such as elections, communal incidents, and public protests. In this paper, we introduce a dataset in English to model political aggression. The dataset comprises public tweets collated across the time-frames of two of the most recent Indian general elections. We manually annotate this data for the task of aggression detection and analyze this data for aggressive behaviour. To benchmark the efficacy of our dataset, we perform experiments by fine-tuning pre-trained language models and comparing the results with models trained on an existing but general domain dataset. Our models consistently outperform the models trained on existing data. Our best model achieves a macro F1-score of 66.66 on our dataset. We also train models on a combined version of both datasets, achieving best macro F1-score of 92.77, on our dataset. Additionally, we create subsets of code-mixed and non-code-mixed data from the combined dataset to observe variations in results due to the Hindi-English code-mixing phenomenon. We publicly release the anonymized data, code, and models for further research.

## introduction in recent years, social media has risen as one of the most popular ways in which people share opinions with each other @xcite . on such platforms, anonymity is a major factor that impacts user behavior @xcite , and the possibility of posting anonymously on platforms such as twitter and reddit has changed the way people communicate @xcite . this has given rise to a significant amount of aggressive behavior-including 1 https://doi.org/10.5281/zenodo.7540489 but not limited to the use of snide remarks, abusive words, and personal attacks, going as far as rape threats @xcite . modern definitions of human aggression establish it as any behavior enacted with the intention of harming another person who is motivated to avoid that harm @xcite . aggression is now defined as social behavior patterns, and several studies have noted the proliferation of abusive language and an increase in aggressive content on social media @xcite . such behavior begets the automated analysis of social media content for aggressive behavior, lying at the intersection of natural language processing (nlp) and computational social sciences (css). nlp research community has proposed various tasks to analyze aggressive behavior, some of which are well-known, viz., offensive language identification @xcite , hate speech detection @xcite @xcite , aggression detection @xcite , cyber bullying @xcite , and so on. various shared tasks have been organized for these nlp sub-areas, motivating us to investigate the phenomenon of aggression on social media. aggression is displayed not only by unnamed and anonymous troll accounts but, on occasion, by known personalities who can influence thousands of followers @xcite . however, we investigate this problem in the context of political trolling and aggression displayed on social media close to the government election. in this paper, we investigate the task of aggression detection on a social media platform, i.e., twitter, in the context of indian elections. we curate a set of political-themed tweets from the user handles of known personalities (∼ 110 in number) and perform manual annotation to create a dataset for the task. our annotation schema aligns with oag he will kill 22000, will abolish nrega, nullify food security. no control on the present. @xcite baba derives satisfaction in being astro-baba. ## datasets for our experiments, we collected two datasets as described below. ## approach recently, sequence classification via fine-tuning of pre-trained language models has become a standard approach for performing various nlp tasks. we take a similar approach and fine-tune various pre-trained language models for the task of aggression detection to report the results below. we select some monolingual, some multilingual, and some pre-trained language models specific to hindi-english code-mixing. every sentence/tweet containing a sequence of words is tokenized into a sequence of sub-words using the model-specific tokenizer. the input to the model is a sequence of sub-word tokens that pass through the transformer encoder layers. the output from the transformer is an encoder representation for each token in the sequence. we take the encoder representation of the [cls] token in the case of bert or the last encoder hidden states for other models. the output layer is a linear layer followed by softmax function, which takes in the above representation. the model is trained by optimizing for the cross-entropy loss value. ## experimental setup we fine-tune various pre-trained languages (both monolingual and multilingual) for the task of aggression detection and use the following pre-trained language models: bert @xcite , roberta @xcite , albert @xcite which are pre-trained over english data. we also include xlm-roberta @xcite , both base and large variants, which are trained over multilingual data (containing both english and hindi) and muril @xcite , a multilingual language model specifically built for indian language. finally, hingbert, hingmbert, and, hingroberta @xcite are also included as they are pretrained over code-mixed hindi-english data. ## results and discussion we report the results obtained via fine-tuning pretrained language models in this section. ## conclusion and future work in this paper, we curate a novel dataset to model political aggression. we analyze this dataset using various approaches like topic modeling, aggression detection, and report results. to benchmark our performance, we also perform the aggression detection task with the help of an existing dataset. our results and analyses also take into account the code-mixing phenomenon observed on social media platforms. the zero-shot cross-dataset experiments show the efficacy of our dataset, which consistently outperforms the approaches used with existing data. while political aggression is subtle occasionally, we observe that some data instances show overtly aggressive behavior. it is important to note the limitations of such a study and we discuss them in the next section. we release any data, code, and models produced during this study (including any raw data, but keeping user handles anonymous) publicly for further research by the community. we license this release under cc-by-nc-sa 4.0. in the future, we aim to collect more data from multiple social media platforms and release it to model aggressive behavior. we plan to perform similar experiments on a large dataset while benchmarking and comparing our current models' performance. we also plan to investigate online or active learning for the same. finally, we also aim to expand on the theoretical underpinnings of sublime aggression and offense by attempting to identify these within other more tangential domains, viz., comedy.