Towards Safer Communities: Detecting Aggression and Offensive Language in Code-Mixed Tweets to Combat Cyberbullying
Year: 2023

Abstract:
Cyberbullying is a serious societal issue widespread on various channels and platforms, particularly social networking sites. Such platforms have proven to be exceptionally fertile grounds for such behavior. The dearth of highquality training data for multilingual and lowresource scenarios, data that can accurately capture the nuances of social media conversations, often poses a roadblock to this task. This paper attempts to tackle cyberbullying, specifically its two most common manifestations aggression and offensiveness. We present a novel, manually annotated dataset of a total of 10, 000 English and Hindi-English codemixed tweets, manually annotated for aggression detection and offensive language detection tasks1. Our annotations are supported by interannotator agreement scores of 0.67 and 0.74 for the two tasks, indicating substantial agreement. We perform comprehensive fine-tuning of pre-trained language models (PTLMs) using this dataset to check its efficacy. Our challenging test sets show that the best models achieve macro F1-scores of 67.87 and 65.45 on the two tasks, respectively. Further, we perform cross-dataset transfer learning to benchmark our dataset against existing aggression and offensive language datasets. We also present a detailed quantitative and qualitative analysis of errors in prediction, and with this paper, we publicly release the novel dataset, code, and models.

text: The 7th Workshop on Online Abuse and Harms (WOAH), pages 29–41 July 13, 2023 c©2023 Association for Computational Linguistics

heading: 1 Introduction
text: Social media is a group of Internet-based applications that allows the creation and exchange of usergenerated content. Lately, it has risen as one of the most popular ways in which people share opinions with each other (Pelicon et al., 2019). With rapid advances in Web 3.0, social media is expected
1https://github.com/surrey-nlp/ woah-aggression-detection/blob/main/data/ New10kData/cyberbullying_10k.csv
OAG You wont march against kids being raped in country or the endless stream of migrants maybe cheaper energy no but you would march against @username pathetic fking pathetic!
CAG Wait a few days sir, you are getting used to hearing harsh words. In future, when all the banks will be of Adani or Ambani, then you will also have to listen to abuses for withdrawing your deposits. #demonetisation
NAG Well this is pure goosebumps, whenever I see him I feel so proud that he is our PM, a living legend for sure
Table 1: Examples of overtly aggressive (OAG), covertly aggressive (CAG), and non-aggressive (NAG) tweets from our dataset.
to evolve and emerge as an even more vital and potent means of communication. Simultaneously, there has also been noticed a sharp uptick in bullying behavior - including but not limited to the use of snide remarks, abusive words, and personal attacks, going as far as rape threats (Hardaker and McGlashan, 2016) on such platforms. In this context, by leveraging the technological advancements in machine learning and natural language processing, automatic detection of instances of cyberbullying on social media platforms such as Twitter can help create a safer environment. Here we investigate two forms of cyberbullying - aggression and offensiveness.
Aggression has been defined as any behavior enacted with the intention of harming another person who is motivated to avoid that harm (Anderson et al., 2002; Bushman and Huesmann, 2014). Several studies have noted the proliferation of abusive language and an increase in aggressive content on social media (Mantilla, 2013; Suzor et al., 2019)
29
OFF Bhikaris like u, can u first afford watching movie in threatres? Talk about that first. Just coz internet is cheap does not mean u will do open defecation in social media. MDRCHD bhaag BSDK
NOT Who doesn’t enjoy the daily press briefings? They really ease the tension! We have to find some way to keep ourselves entertained
Table 2: Examples of offensive (OFF) and nonoffensive (NOT) tweets from our dataset.
On the other hand, offensiveness has been described as any word or string of words which has or can have a negative impact on the sense of self or well-being of those who encounter it (MolekKozakowska, 2022) – that is, it makes or can make them feel mildly or extremely discomfited, insulted, hurt or frightened.
Motivation The dearth of manually-annotated datasets for the tasks of aggression detection and offensive language detection, especially in the HindiEnglish code-mixed setting, necessitated us to work in this area.
This paper investigates the tasks of aggression detection and offensive language detection on Twitter data. We curate politically-themed tweets and perform manual annotation to create a dataset for the tasks. Our annotation schema is in line with the existing aggressive and offensive language detection datasets. With the help of pre-trained language models, we fine-tune pre-trained language models for both tasks and discuss the obtained results regarding precision, recall, and macro F1-scores. The key contributions of this work are:
• Introduction of a novel, manually-annotated dataset containing English and Hindi-English code-mixed tweets to model aggression and offensiveness in text.
• Validation of our dataset’s efficacy for aggressive and offensive language detection tasks within two subsets of this data, viz., English and Hindi-English code-mixed.
• Cross-validation of dataset efficacy with the help of zero-shot transfer learning-based experiments on existing datasets.
• Quantitative and qualitative analysis of erroneous predictions.

heading: 2 Related Work
text: Our work deals with two different but correlated classification tasks. In the available literature, both have been investigated with the help of various machine learning and deep learning-based methods. Below, we provide a detailed overview of the literature from both tasks in separate subsections.

heading: 2.1 Aggression Detection
text: We model aggression detection as a multi-class classification task where our schema is defined as proposed in the TRAC dataset (Kumar et al., 2018a). However, the earliest approaches used decision trees (Spertus, 1997a) to detect aggression with the help of manual rules. These rules were based on syntactic and semantic features. Later, the focus shifted to feature engineering from text which included features like Bag-of-Words (BOW) (Kwok and Wang, 2013; Liu et al., 2019a), N-grams at the word level (Pérez and Luque, 2019; Liu and Forss, 2014; Watanabe et al., 2018), Ngrams at the character level (Gambäck and Sikdar, 2017; Pérez and Luque, 2019), typed dependencies (Burnap and Williams, 2016b), part-of-speech tags (Davidson et al., 2017b), dictionary-based approaches (Molek-Kozakowska, 2022) and lexicons (Burnap and Williams, 2016b; Alorainy et al., 2019).
Word embedding-based approaches for automated extraction of these features from text further improved the detection of aggressive text (Nobata et al., 2016; Zhang et al., 2018; Badjatiya et al., 2017; Kshirsagar et al., 2018; Orăsan, 2018; Pratiwi et al., 2019; Galery et al., 2018). Various deep learning-based architectures proposed in the literature use word embeddings to encode features in the text (Nina-Alcocer, 2019; Ribeiro and Silva, 2019). Authors have proposed the use of Convolutional Neural Networks (Gambäck and Sikdar, 2017; Roy et al., 2018; Huang et al., 2018), LongShort Term Memory (LSTM) (Badjatiya et al., 2017; Pitsilis et al., 2018; Nikhil et al., 2018), Gated Recurrent Unit (GRU) (Zhang et al., 2018; Galery et al., 2018), or a combination of different Deep Neural Network architectures in an ensemble setting (Madisetty and Sankar Desarkar, 2018).
However, state-of-the-art performance (Bojkovský and Pikuliak, 2019; Ramiandrisoa and
Mothe, 2020; Mozafari et al., 2019) was achieved with the help of pre-trained language models (PTLMs) with encoders like ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019). Further, we also observe the use of these contextual embeddings in SemEval-2019 Task 6 (Zampieri et al., 2019b) for English tweets, and TRAC (Kumar et al., 2018b) for Hindi and English tweets and Facebook comments; further motivating us to explore the use of multiple pre-trained language models to validate the efficacy of our dataset.

heading: 2.2 Offensive Language Identification
text: We model the offensive language identification task as a binary classification problem. Waseem et al. (2017) proposed a typology for abusive language and synthesize the typology with two-fold siders containing whether the abuse is ‘generalized’ or ‘directed’ vs. when it is ‘explicit’ or ‘implicit’. They discuss the distinction between explicit and implicit in the context of ‘denotation’ vs. ‘connotation’ as discussed by (Barthes, 1957). A detailed review of hate speech detection (Schmidt and Wiegand, 2017) task has surveyed various approaches deployed in the past. Spertus (1997b) propose a rule-based framework for identifying hostile messages where they use manually constructed rules to identify profanity, condescension, insults and so on. Razavi et al. (2010) utilize a flame annotated corpus which contains a lexicon of hostile and abusive words to detect offensive language in personal and commercial communication. Dictionaries (Liu and Forss, 2015) and bag-of-words (Burnap and Williams, 2016a) have also been proposed as lexical features to detect offensive language.
The use of machine learning algorithms to detect offensive language has been prevalent in the research community (Davidson et al., 2017a; Waseem and Hovy, 2016). Further, the use of word embeddings learned with the help of word2vec or FastText approaches combined with machine/deep learning improved the performance of offensive language identification by a significant margin (Rakib and Soon, 2018; Herwanto et al., 2019; Badri et al., 2022). However, as we point out in the previous subsection, state-of-the-art performance has been achieved with the help of PTLMs.
Pitenis et al. (2020) perform the task specifically for the low-resource Greek language. Similarly, Ranasinghe and Zampieri (2020) show that the use of cross-lingual embeddings for inter-task
and inter-language scenarios is beneficial. The authors first train a multilingual PTLM (XLM-R) on the English data, and then further continue the training using saved weights and softmax layer, for other languages viz. Hindi, Bengali, and Spanish.
Further, there have been a lot of efforts to create datasets for the detection of offensive language and hate speech2 on social media. Çöltekin (2020) presents a dataset for the Turkish language with a specified target for offense. Díaz-Torres et al. (2020) build the same for Mexican Spanish. A clear majority of studies deal with the English language. While other Indian language datasets have been proposed, there is a clear dearth of EnglishHindi datasets which also address code-mixing, in the available literature (Chakravarthi et al., 2021, 2022) except a few (Mathur et al., 2018; Saroj and Pal, 2020).

heading: 3 Dataset Creation
text: We create a dataset containing a mix of English and Hindi-English sentences, to ensure that sufficient data is available for our research. We used the official Twitter API to obtain data from Twitter. Initially, we collected 15,000 tweets based on the search results for one of the 52 keywords (listed in Table 10 in Appendix) in our list pertaining to recent political events and popular political personalities.
We filtered out tweets that were in any language other than English or Hindi (or containing a mix of both) using XLM Roberta-base with a classification head on top (Conneau et al., 2020). Next, with the help of HingBERT-LID code-mixed language identification model (Nayak and Joshi, 2022), we created subsets of tweets belonging to one of the two aforementioned categories.
We preprocessed the tweets by masking all usernames to minimize the introduction of bias to the annotators. Finally, after cleaning, we were left with 5,452 English monolingual and 4,548 HindiEnglish code-mixed tweets.

heading: 3.1 Annotation Setup
text: The following guidelines were supplied to the annotators, which outline the definition and provide a few sample tweets for each Aggression and Offensive Language label.
Task I: Aggression Detection 2hatespeechdata.com - a catalog of hate speech datasets
Aggression focuses on the user’s intention to be aggressive and harmful, or to incite, in various forms, violent acts against a target. The aggression level in the text is categorized into three classes:
• Overtly Aggressive (OAG): This type of aggression shows a direct verbal attack pointing to a particular individual or group.
For example, in the sample tweet for OAG in Table 1, the person expresses frustration over issues such as child sexual abuse, immigration, and high gas prices while also condemning the apathy of others towards these issues. The aggression here is overt, as also seen by the use of words “fking” and “pathetic” in the tweet.
• Covertly Aggressive (CAG): In this type of aggression, the attack is not direct but hidden, subtle, and more indirect while being stated politely in most cases.
For example, in the sample tweet for CAG in Table 1, the person harbors angst against the process of demonetization of the Indian currency and privatization of banks, but chooses to display it covertly while conversing over Twitter.
• Not Aggressive (NAG): Generally, these types of text lack any kind of aggression. It is used to state facts, express greetings and good wishes occasionally, and show agreeableness and support.
For example, in the sample tweet for NAG in Table 1, the person does not display any aggression at all - on the contrary, they praise the PM by calling them a “living legend”.
Task II: Offensive Language Detection
Offensiveness focuses on the potentially hurtful effect of the tweet content on a given target. Text can be identified as belonging to either of the two offensiveness classes:
• Offensive (OFF) This category of text often contains offensive words such as sarcastic remarks, insults, slanders, and slurs.
For example, in the sample tweet for OFF in Table 2, the person uses words such as “bhikaris” (“beggars”) for others, while also availing outright derogatory Hindi slang to address them.
• Not Offensive (NOT) In this category, there is either a thorough use of positive and uplifting language, such as salutations or homage, or a neutral tone.
For example, in the sample tweet for NOT in Table 2, the person makes a remark about how everybody enjoys the daily press briefings, and how they ease tension and keep everybody entertained. There is no offensive tone in this instance.
Setup Our team of annotators consisted of two undergraduate students fluent in both Hindi (native) and English as their second language. The selection of annotators was objective and unbiased. The aforementioned guidelines were made available to them, to refer to while deciding upon the labels for the tweets. This was done to ensure that their political beliefs/loyalties do not play a role in the annotation process. We also recorded their highest level of education and medium of schooling to ensure that the annotations would be of the desired quality, and we informed them about the collection of this data.
All usernames in the data were masked, so at any given point, only the tweet content was visible to the annotators whereas the target personality/organization was hidden from their purview. To ensure the confidentiality of data and to check biases, any metadata too, such as the tweet senders’ demographic identity, was not made available to the annotators.
Moreover, since the tweets often contained aggressive and highly abusive language, the annotators were also given a choice to quit whenever they felt uncomfortable with the task.

heading: 3.2 Inter Annotator Agreement
text: While labeling, each annotator had to decide independently which category the comment belonged to, with the help of a set of guidelines. It can be inferred that all the annotators clearly understood
the guidelines for annotation, as in most cases, they arrived at the same annotation freely. To quantify how good the annotation decisions were, we calculated Cohen’s Kappa score to measure the inter-annotator agreement. It may be noted that a high score on this statistical metric does not mean the annotations are accurate. It only shows the homogeneity of agreement among the annotators about the chosen label.
We obtained an agreement score of 0.67 for Task I, and a score of 0.74 for Task II, both of which indicate “substantial agreement” (p >0.05). In case of disagreement on any instance, we obtained a label on such instances with the help of a third annotator.

heading: 3.3 Dataset Statistics
text: Table 3 shows the exploratory statistics on our dataset for aggression and offensiveness, respectively. We have a total of 10, 000 data instances in the form of tweets. Out of this, 2, 284 are overtly aggressive (OAG), 3, 073 are covertly aggressive (CAG), and 4, 679 are not aggressive (NAG). Similarly, there are 3, 072 offensive (OFF) and 6, 928 not offensive (NOT) instances in the dataset.
Additionally, the monolingual vs. code-mixed statistics are also mentioned for each class in both tables. We have 1, 134 monolingual and 1, 150 code mixed tweets in the OAG category, 1, 715 monolingual and 1, 322 code mixed tweets in the CAG category, and 2, 599 monolingual and 2, 080 code mixed tweets in the NAG category. Similarly, there are 1, 323 monolingual and 1, 749 code mixed tweets in the OFF category and 4, 125 monolingual and 2, 803 code mixed tweets in NOT.

heading: 4 Approach
text: In recent times, sequence classification via finetuning of pre-trained language models has become a standard approach for performing various NLP tasks. We take a similar approach and fine-tune some pre-trained language models for the two tasks, and report the results in the next section. We work with two general-purpose English models, one multilingual model, one model trained specifically on Hindi-English code-mixed data, and one trained exclusively on Twitter data.
Every tweet containing a sequence of words is tokenized into a sequence of sub-words using the model-specific tokenizer. This sequence of subword tokens is the input to the model that passes
through the Transformer’s encoder layers. An encoder representation for each token in the sequence is the output from the transformer. We take the encoder representation of the [CLS] token in the case of BERT or the last encoder hidden states for other models. The output layer is a linear layer followed by the softmax function, which takes in the above representation. The model is trained by optimizing for a custom weighted cross-entropy loss value which we explain in detail in an upcoming subsection.
Experimental Setup We fine-tune for the aforementioned two tasks the following pre-trained language models: BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019b) which are trained on English data, XLM-R (Conneau et al., 2020) base which is trained over multilingual data containing both Hindi and English, HingRoBERTa (Nayak and Joshi, 2022), a multilingual language model specifically built for Hindi-English code-mixed language as seen in the Indian context, and Bernice (DeLucia et al., 2022), a multilingual language model trained exclusively on Twitter data.
Data Split and Evaluation Criteria We report macro F1-scores on our complete dataset, as well as on its code-mixed and non-code-mixed subsets individually. For the train/validation/test splits, we choose uniform 80% / 10% / 10% from each dataset to perform the experiments.
Experiment Settings We perform experiments using the Huggingface Transformers library (Wolf et al., 2020). We monitor the validation set’s macro F1-scores to find the best hyperparameter values, using the following range of values for selecting the best hyperparameter:
• Batch Size: 8, 16, 32
• Learning Rate: 1e-5, 1e-6, 3e-5, 3e-6, 5e-5, 5e-6
We repeat each training five times with different random seeds and report the mean macro F1scores along with their standard deviation. Our experiments were performed using 2 x Nvidia RTX A5000 and a single training run usually takes approximately 1 hour on the dataset. For the subsets, however, the runtime is approximately 30 minutes. The models generated during our experiments see the number of trainable parameters varying from 100M to 200M depending upon the language model used.
Custom Weighted Loss As our dataset exhibits class imbalance, we use weighted cross-entropy loss (Lee and Liu, 2003) in all our experiments. We assign a weight to the loss of every instance depending on the class label. Then, we find the percentage of examples by class belonging to each class from the train split and take the inverse of the probability values as the weight for the particular class. In this way, we give more importance to the instances belonging to the minority class.

heading: 5 Results
text: We report the results obtained via fine-tuning pretrained language models in this section. Table 4 reports the test set macro F1-scores from pre-trained language models for the two tasks of aggression detection and offensive language detection on our dataset. In addition to this, we also present the scores on English monolingual and Hindi-English code-mixed subsets of our dataset.
For aggression, we observe that XLM-Rbase outperforms other pre-trained language models on our overall dataset, achieving the highest macro F1score of 67.87. On the English subset, we observe that RoBERTabase performs better than other models with a macro F1-score of 66.63, whereas for the Hindi-English code-mixed subset, Hing-RoBERTa
gives the best macro F1-score of 66.94. For offensive language detection, we observe that Hing-RoBERTa outperforms other pre-trained language models on our overall dataset, achieving the highest macro F1-score of 65.45. On the English subset, we observe that RoBERTabase outperforms other models with a macro F1-score of 63.46. For the Hindi-English code-mixed subset, Hing-RoBERTa once again gives the best performance with a macro F1-score of 64.97.
Cross-dataset Transfer Learning We perform transfer learning experiments to benchmark our dataset against some existing datasets for the same tasks. Results from our transfer learning setup are presented in Table 5.
For the task of aggression detection, we benchmark our dataset against a curated subset of the TRAC (Trolling, Aggression, and Cyberbullying) dataset (Bhattacharya et al., 2020). This subset, (discussed in Table 8 in section 9), contains instances in Hindi (Roman script) and English, and is annotated for aggression (OAG: overtly aggressive, CAG: covertly aggressive, NAG: not aggressive). For the task of Offensive language detection, we use OLID (Offensive Language Identification Dataset) (Zampieri et al., 2019a) to benchmark our dataset. OLID is an English language dataset and we make use of its Level-A labels (OFF: offensive, NOT: not offensive), discussed in Table 9 in section 9. We chose these two datasets because their annotation schema aligned with that of ours, for aggression detection and offensive language detection tasks respectively.
For Aggression detection, the columns D1–>D2 and D2–>D1 in Table 5 present a cross-dataset setup within which we observe the performance of models fine-tuned on D1 (our dataset) and tested on D2 (TRAC dataset), and vice versa. We observe
Tweets | Task: Aggression Detection GT M1 M2 M3 Error Cause
Romanticizing open defecation under heavy rain to enjoy the melancholy
CAG NAG CAG NAG Sarcasm
@username The way Rahul Gandhi changed his DP to Nehru holding a tricolour, I want to change it to Savarkar or Golwalkar holding the Flag. Can anyone help and share pictures of theirs holding the tricolor..
CAG NAG NAG NAG Real-world context
@username You use words like waqf, muslims, mullahs, terrorists, radicals and you will get a block message from twitter, hope Elon buys twitter very soon. CAG CAG NAG NAG Hidden Aggression
Table 6: Prediction on test set instances from resultant models for aggression detection. GT: Ground Truth label, M1: XLM-Rbase, M2: RoBERTabase, M3: Hing-RoBERTa.
that models trained on our dataset obtain better F1scores than those trained on the TRAC dataset. Further, we observe that the best performance achieved in this setup is with the help of XLM-Rbase, the same multilingual model which also performs the best on the combined dataset in Table 4.
For Offensive language detection, we examine the results in columns D1–>D2 and D2–>D1 in Table 5, which note the performance of models finetuned on D1 (our dataset) and tested on D2 (OLID dataset), and vice versa. As was true with the first task, it is observed here too that models trained on our dataset obtain better F1-scores as compared to the models trained on the OLID dataset. Additionally, we observe a similar correlation between both sets of results. The model fine-tuned on codemixed data, Hing-RoBERTa, performs the best in this scenario, as was the case with the combined dataset performance in Table 4.
The overall decrease in F1-scores observed across models for the Offensive language detection task can be attributed to the dissimilarities in the composition of the OLID dataset and our dataset, despite both being annotated for the offensive language identification task with the same annotation schema. While our dataset contains English and Hindi-English tweets pertaining specifically to the Indian political scenario, OLID is an English-language dataset with no instances of Hindi-English code-mixing, and little to no emphasis on regional or national politics.
On the contrary, the TRAC dataset contains English and Hindi-English sentences with a clear focus on the conversational data generated within India, which explains why we see a greater harmony in Table 5 between the TRAC dataset and our dataset, as compared to the OLID dataset.
Error Analysis
For error analysis, we pick the best-performing models for monolingual, code-mixed, and combined datasets, which as per our experiments have been RoBERTabase, Hing-RoBERTa, and XLMRbase respectively. We report some of the most common error patterns in Table 6 and Table 7.
For the task of aggression detection, instances carrying sarcasm that make heavy use of oxymoronic/ironic language were misclassified the most by all three models. An example of this is the first tweet in Table 6, where the person who made the tweet observes discontent with the practice of open defecation not by attacking it directly but with sarcasm. Another common error we observed was among instances, that seemed to have a neutral tone ostensibly but required some real-world knowledge to understand the context of aggression within. The second tweet in Table 6 is an excellent example of this. By itself, the tweet does not appear to be aggressive, but its true meaning unveils when read along with context. A few wrongful predictions can also be observed because of the aggression being very covert or hidden, as seen in the third tweet in Table 6 where under the garb of advocating for Elon Musk’s free speech, the person is expressing an intent to, in fact, be disrespectful and use words on the platform that spread disharmony.
For the task of offensive language detection, the most common error type was observed due to the presence of offensive named entities. The first tweet in Table 7 is an example of this, where the use of “Khujliwal” (a pun on “Kejriwal” - which is the name of an Indian politician), is the cause of offense, as labeled by our annotators. Another common error was in instances that required real-world knowledge to understand their full context. For
Tweets | Task: Offensive Language Detection GT M1 M2 M3 Error Cause
@username It was always very clear that Khujliwal is a Godse Lover
OFF NOT NOT NOT Named entity
@username @username Dogs are at least loyal bro ..not these rice bags OFF NOT NOT NOT Real-world context
@username @username Nah, you need to do Ghar Wapasi to find real Moksha. Else you will remain a mlechha OFF NOT NOT OFF Code-mixed
Table 7: Prediction on test set instances from resultant models for offensive language detection. GT: Ground Truth label, M1: XLM-Rbase, M2: RoBERTabase, M3: Hing-RoBERTa.
example, in the second tweet in Table 7, “rice bags” is actually a derogatory slur used quite commonly in the Indian political context. Finally, we also observe misclassified instances due to the code-mixed nature of tweets, as seen in the third tweet in Table 7 where the word “mlechha” has derogatory connotations.

heading: 6 Conclusion and Future Work
text: This paper presents a novel dataset to model aggressiveness and offensiveness in text. We analyze this dataset using approaches such as fine-tuning pre-trained language models for the task of aggression detection and offensive language detection and report the results. Our analysis also takes into account the code-mixing phenomenon observed on social media platforms as we report additional results for this task. Since aggression and offense can be subtle, and their identification in the text can sometimes be subjective, it is important to note the limitations of such a study - which we discuss in the next section. We release any data (including any raw data, but only in the form of tweet IDs and their respective labels for the two tasks), code, and models produced during this study publicly for further research by the community. We license this release under CC-BY-SA 4.0.
In the near future, we aim to annotate this data for tasks such as sarcasm detection - to develop a deeper understanding of how it is related to aggression and offensiveness. Additionally, the motivation for collecting the same data instances marked with aggression and offense labels is for a multitask learning-based model also to be able to identify when 1) the tone of a text is aggressive without being offensive vs., 2) the text is offensive, despite it not being overtly aggressive. We also aim to collect more data and annotate it using weak supervision. Finally, we also aim to expand on the theoretical
underpinnings of sublime aggression and offense by attempting to identify these within other more tangential domains, viz., comedy.

heading: 7 Limitations
text: Our work can be considered to have the following limitations:
1. The dataset we introduce contains 10, 000 text instances sampled from a single social media platform. However, we acknowledge this limitation and as noted in section 6, we aim to extend this work by collecting more political data across various social media platforms and using it to model aggressive behavior.
2. We obtained this dataset by crawling for tweets based on 52 keywords (as shown in Table 10). We acknowledge that these keywords may have limited the domains in which political aggression can occur. That being said, we also hope that task generalizability is not compromised due to the presence of pretrained language models at the helm of our experiments.

heading: 8 Ethics Statement
text: Our dataset of tweets was obtained by scraping Twitter. All tweets have been anonymized, and metadata such as senders’ demographic identity is never included in the data used to train our models. We plan to release only the tweet ids and their respective labels for the two tasks as part of our dataset.

heading: 9 Appendix
text: We note the language-wise class distribution for aggression and offensiveness classes, in the publicly available TRAC and OLID datasets respectively, in Table 8 and Table 9. Next, we list the keywords used for data scraping, during the creation of our novel dataset.
TRAC Dataset Statistics Table 8 shows the exploratory statistics on TRAC dataset for aggression. There are a total of 9, 800 data instances. Out of this, 2, 172 are overtly aggressive (OAG), 3, 279 are covertly aggressive (CAG), and 4, 349 are not aggressive (NAG).
Additionally, the monolingual vs. code mixed statistics are also mentioned for each class. We have 1, 114 monolingual and 1, 058 code mixed tweets in the OAG category, 1, 693 monolingual and 1, 581 code mixed tweets in the CAG category, and 1, 820 monolingual and 2, 529 code mixed tweets in the NAG category.
OLID Dataset Statistics Table 9 shows the exploratory statistics on OLID dataset for offensive language. There are a total of 9, 532 data instances. Out of this, 3, 168 are offensive (OFF) and 6, 364 are not offensive (NOT).
Additionally, here too we mention the monolingual vs. code mixed statistics for each class. We have 2, 034 monolingual and 1, 134 code mixed tweets in the OFF category and 3, 578 monolingual and 2, 786 code mixed tweets in the NOT category.
Keywords for Scraping Tweets Table 10 contains a list of 52 keywords that were used in the initial scraping of tweets, for creation of our novel dataset. These keywords were obtained from Twitter’s top trending keywords list of the previous two years.

heading: Future Work
text: In the near future, we aim to annotate this data for tasks such as sarcasm detection - to develop a deeper understanding of how it is related to aggression and offensiveness. Additionally, the motivation for collecting the same data instances marked with aggression and offense labels is for a multitask learning-based model also to be able to identify when 1) the tone of a text is aggressive without being offensive vs. , 2) the text is offensive, despite it not being overtly aggressive. We also aim to collect more data and annotate it using weak supervision. Finally, we also aim to expand on the theoretical
underpinnings of sublime aggression and offense by attempting to identify these within other more tangential domains, viz. , comedy