Garain at S em E val-2020 Task 12: Sequence Based Deep Learning for Categorizing Offensive Language in Social Media (2020)

Abstract: SemEval-2020 Task 12 was OffenseEval: Multilingual Offensive Language Identification inSocial Media (Zampieri et al., 2020). The task was subdivided into multiple languages anddatasets were provided for each one. The task was further divided into three sub-tasks: offensivelanguage identification, automatic categorization of offense types, and offense target identification.I participated in the task-C, that is, offense target identification. For preparing the proposed system,I made use of Deep Learning networks like LSTMs and frameworks like Keras which combine thebag of words model with automatically generated sequence based features and manually extractedfeatures from the given dataset. My system on training on 25% of the whole dataset achieves macro averaged f1 score of 47.763%.

## introduction the ever-growing amount of user-generated data on social media platforms be it facebook, twitter, blogs or any other electronic medium introduces new challenges in terms of automatic content moderation, especially regarding hate speech and offensive language detection. not only is hate speech more likely to happen on the internet, where anonymity is easily obtained and speakers are psychologically distant from their audience, but its online nature also gives it a far-reaching and determinative impact @xcite . user content mostly consists of microposts, where the context of a post can be missing or inferred only from current events. manual verification of each posting by a human moderator is infeasible due to the high amount of postings created every day. consequently, automated detection of such attacking postings is the only feasible way to counter this kind of hostility. however, this task is challenging because natural language is fraught with ambiguities, and language in social media is extremely noisy. the classification system that would be prepared for the task, needed to be generalized for various test corpora as well. in this paper i have described the system consisting of a sequential pipeline with text feature extraction and classification as its main components. firstly, a bag-of-words model is used for encoding the sentences into corresponding integer sequence. thereafter, vectors are generated from these sequences and fed to a series of bilstm layers for training. then a softmax layer is used for ternary classification into the corresponding offensive language categories. the rest of the paper has been organized as follows. section 2 describes the data, on which, the task was performed. the methodology followed is described in section 3. this is followed by the results and concluding remarks in section 4 and 5 respectively. ## data the dataset that has been used to train and validate the model is an annotation following the tagset proposed in the offensive language identification dataset (olid) @xcite and has been used in offenseval 2019. this year more accurate and quantified dataset was generated and annotated using a semisupervised approach to increase the quality and quantity of the dataset named the semi-supervised offensive language identification dataset (solid) @xcite . the resulting dataset that is the newer version of the olid dataset has been used for training and validating purposes. it was collected from twitter; the data being retrieved the data using the twitter api by searching for keywords and constructions that are often included in offensive messages. the vast majority of content is not offensive so different strategies have been tried to keep a reasonable number of annotations in the offensive class amounting to around 26% of the dataset. the dataset provided consisted of annotations in their original form along with the corresponding labels. subtask c consisted of the labels ind, grp and oth. ## methodology my approach is to preprocess the annotations and then convert the annotations into a sequence of words and convert them into word embeddings. i then run a neural-network based algorithm on the processed tweet. label based categorical division of data is given in in step 1, for example, ",-) " is replaced by "winking happy" ":-c" is replaced with "real unhappy" ";-(" is replaced with "crying" similarly i replaced 110 emoticons by their feelings. the step 3 consists of manually identifying a feature of inclusion of words "xx" or "xxx" in sexual harassment cases and taking advantage of this identified feature. using this extraction, contributed to increasing accuracy in classification to some extent. they played an important role during the modeltraining stage. the pre-processed annotations are treated as a sequence of words with interdependence among various words contributing to its meaning. i then take a bag-of-words model approach as well as use pre-trained glove vectors. for the bag-of-words approach i convert the annotations into one-hot vectors as shown in fig. 1 . the bag-of-words approach outperformed so i have shown results related to this encoding of text. singular verbs clearly depict individuals while plural verbs depict groups and others, so their counts are included as features along with other manually extracted features listed below: the architecture is depicted in ## results i participated in subtask c of offenseeval 2020 which is task 12 of semeval 2020 and my system worked quite well. i have included the automatically generated macro averaged evaluation metrics along with the detailed metrics calculated using the gold labels. the results are depicted in tables 5 6 . the confusion matrix is shown in fig. 3 . ## conclusion in this paper, i have presented a model which performs satisfactorily in the given task. the model is based on a many2one sequence learning based architecture. there is scope for improvement by including more manually extracted features (like those removed in the pre-processing step) to increase the performance. i