TECHSSN at S em E val-2020 Task 12: Offensive Language Detection Using BERT Embeddings (2020)

Abstract: This paper describes the work of identifying the presence of offensive language in social media posts and categorizing a post as targeted to a particular person or not. The work developed by team TECHSSN for solving the Multilingual Offensive Language Identification in Social Media (Task 12) in SemEval-2020 involves the use of deep learning models with BERT embeddings. The dataset is preprocessed and given to a Bidirectional Encoder Representations from Transformers (BERT) model with pretrained weight vectors. The model is retrained and the weights are learned for the offensive language dataset. We have developed a system with the English language dataset. The results are better when compared to the model we developed in SemEval-2019 Task6.

## introduction the usage of offensive or hate words is increasing these days, largely in online communication. people find it easier to express their opinions and thoughts in online sources rather than do it personally. the anonymity provided by the online environment encourages many people to express their views in aggressively. the information spread rate is extremely fast in online social media. people, without checking the validity of the information they receive, spread it to others. the text content posted in messages, websites, social media and blogs are highly unstructured, informal, often misspelt, and use shorthanded notations, emojis and emoticons. getting the meaning in the natural text is a complicated task. there is ongoing research in the field of offensive language or hate speech detection, yet it remains only a goal to obtain a full-fledged model. we have participated in the offensive language task conducted in semeval-2019 by @xcite with various machine learning and deep learning models in @xcite . for semeval-2020 by @xcite , we have developed and tested deep learning models with different word embeddings. we have used glove @xcite , word2vec @xcite and bert embedding @xcite pretrained models to identify the presence of offensiveness in tweets. we have participated in subtask a (off/not) of classifying whether a tweet is offensive or non-offensive and subtask b (tin/unt) of classifying whether an offensive tweet is targeted to anyone or untargeted. the rest of the paper is organized as follows. section 2 surveys the related work in this field. section 3 describes the methodology used to solve the task. results are discussed in section 4 and conclusion in section 5. @xcite and @xcite . most of the work use the machine learning and deep learning techniques. the work done by @xcite uses bert uncased model with an f1 score of 0.8057 for task a and 0.50 for task b. @xcite uses perspective api and bert cased and uncased models to detect the offensive language with an f1 score of 0.7933 for task a and 0.6817 for task b. semeval-2019 task6 report of @xcite says that machine learning algorithms such as support vector machine (svm), logistic regression, and artificial neural network (ann), and deep learning techniques such as convolutional neural network (cnn), recurrent neural network (rnn), bidirectional long short-term memory network (bilstm), embeddings from language models (elmo), bidirectional encoder representations from transformers (bert), gated recurrent unit (gru) and ensemble techniques have been employed for offensive language detection. out of the 115 participant teams, 70% of the teams used deep learning techniques. in that, 20% teams used ensemble techniques, 11% used cnn, 13% used lstm & bilstm, 10% used rnn & gru, 8% used bert and the remaining used other dl methods. ensemble and bert models gave better results for the top teams when compared to the other techniques. ## system methodology we have participated in semeval-2019 task 6 to identify and categorize english tweets using machine learning and deep learning models @xcite , where it was found that 1d-cnn with glove embeddings and 2d-cnn with word2vec embeddings have performed better compared to other machine learning and deep learning algorithms. for semeval-2020 task 12, in addition to these algorithms, we also used deep learning model with bert embeddings. the proposed system comprises of the following modules: ## results and discussion various deep learning models with different word embeddings are used to detect the presence of offensiveness in the given tweet and to identify whether the tweet is targeted or not.