Pin_cod_ at S em E val-2020 Task 12: Injecting Lexicons into Bidirectional Long Short-Term Memory Networks to Detect T urkish Offensive Tweets (2020)

Abstract: This paper describes a system (pin_cod_) built for SemEval 2020 Task 12: OffensEval: Multilingual Offensive Language Identification in Social Media (Zampieri et al., 2020). I present the system based on the architecture of bidirectional long short-term memory networks (BiLSTM) concatenated with lexicon-based features and a social-network specific feature and then followed by two fully connected dense layers for detecting Turkish offensive tweets. The pin cod ’s system achieved a macro F1-score of 0.7496 for Sub-task A - Offensive language identification in Turkish.

## introduction with the appearance of influential social media platforms, offensive language is becoming more prevalent and visible. harbouring behind a physically invisible author, bullies-either an intimate partner or an absolute stranger to victims-spread abusive, offensive and hateful messages against a particular person or a group of people through many internet platforms. the scope of the victims is broad and especially teenagers, women and immigrants are among targets for bullies. online bullying leads to mental health issues including anxiety disorder, depression, it reduces self confidence, and causes lower academic achievement. computational approaches to identify online bullying, hate speech, offensive and abusive language have gained acceleration and attention as a number of workshops have been organized for this purpose @xcite @xcite towards generic users, women and/or immigrants in english, italian, spanish and german. semeval 2020 task 12 : offenseval 2020 is probably the first workshop for identifying offensive language for turkish language with sub-task a -offensive language identification (c ¸öltekin, 2020). i participated in this sub-task the main goal of which is to segregate offensive posts from not offensive ones (i.e., off for offensive, not for not offensive class). offensive posts comprise insults, threats, and untargeted profanity. non-offensive posts do not contain any offense or profanity. the pin cod 's system for this binary classification sub-task was based on bilstm networks incorporated with various lexicon-based features and the presence of user mentions (e.g. @username) and then followed by two fully connected dense layers. the scripts for preprocessing and the bilstm model can be found here 1 . the current study shows the features obtained from several lexicons and a social-network specific feature increased the performance of the bilstm model with a satisfactory f1-score of 0.7496. this paper is organized as follows: related work in section 2, system description in section 3, results in section 4, error analysis and discussion in section 5, and conclusions in section 6. ## related work a spate of studies investigated hate speech in recent decades @xcite @xcite . the phenomenon of hate speech has been investigated under specific aspects, for instance, cyberbullying @xcite @xcite , offensive language @xcite @xcite , abusive language @xcite , insults and profanity @xcite . a typical natural language processing methodology that deals with hate speech detection involves classifying large volumes of text using supervised machine learning approaches @xcite @xcite . ## system description i followed a supervised learning approach to identify offensive language in turkish. i implemented bidirectional long short-term memory networks using word embeddings on the turkish twitter dataset provided by the organizer in semeval-2020 task 12 offenseval: sub-task a -offensive language identification (c ¸öltekin, 2020). to predict the labels for the given test set consisting of 3528 unlabeled tweets, i only used the provided training set containing 31756 labeled tweets which was split into two parts: 80% for training set and 20% for validation set. ## results for offenseval 2020: sub-task a -offensive language identification in turkish, the architecture was built based on the bidirectional long short-term memory networks concatenated with a social-network specific feature, sentiment feature, lexicon-based features and word embeddings. then, it was followed by two fully connected layers so that the test set consisting of turkish tweets were predicted as either offensive (off) or not offensive (not). i did not use any external datasets apart from the training set provided by the organizer. the official evaluation metric in offenseval 2020 was macro f1-score. the pin cod 's system ranked 23rd among 46 participants shown in ## error analysis and discussion the pin cod 's system detecting turkish offensive tweets achieved a satisfactory result. yet some misclassifications were obtained as presented in a conclusion might be made that an algorithmic model of detecting offensive language cannot be solely limited to detecting bad words or slang words. certain insulting terms might sound differently to a friend or an absolute stranger. other factors such as the use of emojis along with such insulting terms might make a message offensive or not offensive. on the contrary, highly offensive messages might not necessarily include any toxic and hurtful words while they contain some metaphors or metonymies. if the predicting model is restricted to the textual analysis of content, it will likely boost the chances of yielding false negatives and false positives. ## conclusions in this paper, i presented the pin cod 's system i have developed as part of my participation in semeval-2020 task 12: offenseval 2020: multilingual offensive language identification in social media. specifically, i have participated in sub-task a -offensive language identification for turkish language. to bring a solution for this task, i adopted bidirectional long short-term memory neural networks incorporating lexical features from a polarity lexicon, hate speech-related lexicon and a compiled offensive/profane/slang wordlist as well as a social-network specific feature. then, this concatenated layer was fed to two fully connnected dense layers. although a satisfactory result is obtained for this task, i plan to improve the system's performance by adopting knowledge base and more social-network specific features in the future.