LUC at C om MA -2021 Shared Task: Multilingual Gender Biased and Communal Language Identification without Using Linguistic Features (2021)

Abstract: This work aims to evaluate the ability that both probabilistic and state-of-the-art vector space modeling (VSM) methods provide to well known machine learning algorithms to identify social network documents to be classified as aggressive, gender biased or communally charged. To this end, an exploratory stage was performed first in order to find relevant settings to test, i.e. by using training and development samples, we trained multiple algorithms using multiple vector space modeling and probabilistic methods and discarded the less informative configurations. These systems were submitted to the competition of the ComMA@ICON’21 Workshop on Multilingual Gender Biased and Communal Language Identification.

## introduction the introduction of the internet and its democratization in the public sphere has fostered the emergence of many sociological phenomena. this opens the possibility of forming friendly relations and information sharing from online networking platforms @xcite . as the organizers say: "aggression and its manifestations in different forms have taken unprecedented proportions with the tremendous growth of internet and social media." 1 . the challenge comma@icon 2021 is an interesting task in order to automatically discover and understand the pragmatic and structural aspects of such forms of language usage @xcite . our international luc 2 team 2 have worked in 1 https://competitions.codalab.org/ competitions/35482 2 team composed by lia (laboratoire informatique d'avignon/université d'avignon, france), ebsi/université de montréal (québec, canada), cenidet (centro nacional de investigación y desarrollo tecnológico, cuernavaca, méxico) and colmex (the colegio de méxico, cdmx, méxico). such challenge using several approaches mainly without linguistic features. this paper is organized as follows: the section 2 presents some relevant state of the art work related to automatic aggression identification, section 3 describes the methodology and the comma@icon'21 dataset used, section 4 shows the results and finally the section 5 analyzes and discusses the contributions of this paper. ## related work there already exist software designed to identify aggression and cyberbullying in social medias, e.g. cyberpatrol. the main drawback of these systems is that they are based on keyword filtering, which is a limitation because no statistical features of texts are taken into account. further, these keyword filtering methods require manual maintenance. to overcome the limitations of keyword filtering systems, @xcite is one of the former attempts to detect cyberbullying by using statistical features: word frequency, analysis of feelings (use of pronouns in the second person, insults, etc.) and context. @xcite built a system that can detect bullying elements in commentaries of youtube videos. these were classified according to different representative categories (sexuality, intelligence, race and physical attributes). the classification revealed weaknesses and an increase in false positive cases. researchers emphasized the importance of using common sense to understand users' goals, emotions, and relationships, thereby disambiguating and contextualizing language. in @xcite the authors were also interested in a word search method based on a bag of words (bow) system incorporating sentiment and contextual analysis. they build a decision tree that predicted intimidating messages with an accuracy rate of 0.93. the researchers also have developed the chatcoder software to detect malicious activities online @xcite . another study tested a system that allows users of a website to control the messages posted on their web pages: it customized vocabulary filtering criteria using a machine learning method that automatically labeled the contents. this approach had limitations because it was unable to measure relationships between terms beyond a certain semantic level @xcite . @xcite provided a concrete method for detecting online harassment by measuring the score of sent and received messages (and thus their degree of involvement in a conversation) using the hyper-link induced topic research algorithm (hits). the authors also proposed a graphical model that identifies the aggressors and their most active victims. other studies have attempted to go further by seeking to take into account more specific characteristics. @xcite tried to establish a system based on language features characterizing the author's genre of comments on myspace. their results revealed an improvement in the detection of bullying when this information is taken into account. as we can see, recent work defines the means to respond to the cyberbullying phenomenon that is becoming more and more widespread as the use of the web does. the problem of identifying offensive and abusive language is a more difficult task than expected due to the prevalence of 5 factors, according to @xcite , described below: other aspects detected by @xcite corresponds to the heterogeneity in the approach to the problem itself that causes too much noise and confusion, such as the fact of only addressing specific aspects, specific contexts, different definitions for certain terms and / or domain, and finally different sets of assessment. at present, the task of classifying text in "agglutinating" or "morphologically rich" languages, leaves aside classical preprocessing and is replaced by the use of deep neural networks and word embeddings, since they take into account the semantic distance of the words in context, which is very useful in categorization tasks, which is not the case with the classic bag of words methods. a clear example of this is the implementation of fasttext for the turkish language by @xcite . ## methodology and data-sets in this section we present the methodology and the data-sets used in our study. ## results we present our results in two parts. in the first one we show the accuracy of methods on the training set. in the second one, the official f1 measures from the codalab website of the organizers are showed in their "results" page 5 . ## discussion and conclusions this task was very challenging. the corpus (train, developing and test) are very noisy at several levels (syntactic, lexical, semantic, etc.), and the instances presents two or more languages mixed. there are instances that make no sense. by example, the instances from train and developing sets: where the string of "r" of the instance c575.31 represents a string of a dog's emojis, is a good example of semantic noise. what was the logic (if any) of the annotators in assigning the classes (nag,ngen,ncom) to this instance? however, our strategy has obtained the third rank on overall-f1 measure and the second rank on the aggression micro-f1 measure. the methodology that we adopted in this study allowed us to observe in broad strokes the complexity of the challenge. we think that our model may be enriched by other model sentences representation, as followed in @xcite , in order to outperform our results. finally, sentences classified with labels 'aggression', 'genre' or 'communal levels' may be used in other nlp tasks, by example on the summaries generation guided by a specific context and coming from social network text data (torres-moreno, 2014).