T eam_ BUDDI at C om MA @ ICON : Exploring Individual and Joint Modelling Approaches for Detecting Aggression, Communal Bias and Gender Bias (2021)

Abstract: The ComMA@ICON 2021 Shared Task involved identifying the level of aggression and identifying gender bias and communal bias from texts in various languages from the domain of social media. In this paper, we present the description and analyses of systems we implemented towards these tasks. We built systems utilizing Transformer-based models, experimented by individually and jointly modelling these tasks, and investigated the performance of a feature engineering method in conjunction with a joint modelling approach. We demonstrate that the joint modelling approaches outperform the individual modelling approach in most cases.

## introduction social media has revolutionized how people communicate and engage in discourse and debate regarding various issues in society. in india, regional languages have influenced how content is generated on social media, with english not being the only language in which people interact with each other on forums. however, it is vital to ensure that discourse on social media is civil and respectful and does not serve as an outlet to malign or abuse users. abusive or hostile content can manifest in various forms, including but not restricted to aggressive or hostile personal comments or posts, content that may be communal and malign religious sentiments or content that may be discriminatory based on gender. therefore it is imperative to handle these types of content in a time-bound and sensitive manner. modelling such text could help build automated or human-in-the-loop systems that can assist manual content moderators in reviewing and flagging such objectionable content. natural language processing can be extremely integral in this regard. however, modelling text * equal contribution from the social-media domain comes with challenges that need to be addressed. first of all, text generated on social media is significantly different from the text in books or newspapers. one such difference is the comparatively short length of texts in social media, such as tweets. another difference would be the informal nature of discourse on social media forums, which includes the usage of slang, emojis and hashtags. thirdly, social media text may be code-mixed, further complicating the process. an example of this is hinglish, a combination of hindi and english. these factors must be considered when modelling such text. ## about the task the comma project's shared task on multilingual gender biased and communal language identification (kumar et al., 2021a) foot_0 foot_1 provided datasets spanning hindi, bangla (indian variety), meitei and english @xcite . the shared task comprised 3 sub-tasks which involved detecting the level of aggression, the identification of gender bias, and the identification of communal bias in a given text. ## challenges since the task involved data from informal domains of discourse like social media, some factors were to be considered while building systems for these tasks. some of those considerations were: ## system overview we built systems towards solving the three subtasks, for the hindi corpus and the multilingual corpus, considering the factors mentioned above. to this end, for tackling the hindi corpus, we utilize a bert @xcite model, which was finetuned on hinglish tweets with the language modelling (lm) task @xcite (kasliwal and bhange) (meghanabhange/hinglish-bert), hereafter referred to as hinglish-bert, as our starting point for all systems we submitted for the hindi task. we utilize xlm-roberta (xlm-r) @xcite (hugging face -xlm-roberta-base) as our starting point for the system built as part of our submission towards the multilingual corpus as hindi, bengali, and english are part of the list of languages used for training the xlm-r model. we are given a multi-task text classification dataset given by @xmath0, y (i) ) n @xmath1, where x is the set of all preprocessed texts in the dataset and @xmath2. also, for a given sample (x, y) ∈ d and task @xmath3, gen, com }, we define y t as the true class annotation corresponding to the task t. we first preprocess each of the text in the raw dataset using the preprocessing steps from @xcite to obtain the preprocessed texts @xmath4. we then embed each of these preprocessed texts @xmath5. for the xlm-r model, we use the representation of the s token as the representation of the text. we now define the function hinglish-bert which embeds a given preprocessed text x into a hidden representation h x as described above. for each of the tasks, @xmath6, gen, com }, we then use task-specific head layers h t to obtain the prediction probabilities ŷt for each of the classes in the task from the hidden representations of the texts as given by: ## experimental setup we utilize pytorch foot_3 @xcite and the transformers library 4 from hugging face @xcite for implementing our systems. the code and resources used are made available on github foot_4 . the systems we built and their respective notations are summarized in ## results for the hindi set, we initially performed one run of each of the systems and submitted the results to the leaderboard. the scores for these runs are present in the first row of tables 9, 10, 11 and 12 foot_6 . in these runs, we observed that the hin-jnt-4h system performed the best, followed by the hin-jnt-3h system. we further re-ran the systems four more times, with different seeds for each run to account for the impact of randomness in our systems' performances. in terms of instance-f 1 , we observe that the joint modelling approaches often outperform the system of individually trained models across the runs. this is also evident in the mean scores reported in however, when we further compare the performance within the different joint modelling approaches, we observe no clear winner under all circumstances, as the performance often varies with the random seed used as part of the run. for the multilingual set, we submitted only one system, multi-jnt-3h, whose results are presented in