IRL ab_ DAIICT at S em E val-2020 Task 12: Machine Learning and Deep Learning Methods for Offensive Language Identification (2020)

Abstract: The paper describes systems that our team IRLab_DAIICT employed for shared task OffensEval2020: Multilingual Offensive Language Identification in Social Media shared task. We conducted experiments on the English language dataset which contained weakly labelled data. There were three sub-tasks but we only participated in sub-tasks A and B. We employed Machine learning techniques like Logistic Regression, Support Vector Machine, Random Forest and Deep learning techniques like Convolutional Neural Network and BERT. Our best approach achieved a MacroF1 score of 0.91 for sub-task A and 0.64 for sub-task B.

## introduction since the past few years, rapid improvement in communication technologies has led to an exponential rise in the number of people who have unrestricted access to the internet. this newfound abundance has seen the emergence of the "netizen", and social media platforms like twitter, facebook, instragram etc, the cyber-agoras where netizens assemble to discuss and debate ideas. but in several cases the exchange on these platforms is not civil and at times the behaviour exhibited by individuals on social media platforms can be hostile, targeting individuals or communities by posting insults or threats. twitter receives more than 500 million tweets daily 1 , which can be classified as offensive or hate speech. this makes the detection of offensive and hateful content on social media platforms an important research question in the field of natural language processing. to promote research in this field several competitions have been organised such as offenseval 2019 @xcite , hasoc 2019 @xcite , hateval 2019 @xcite . offenseval 2020 @xcite task was organised for five languages english, arabic, danish, greek and turkish. the organizers proposed a hierarchical task which consists of three sub-tasks as below: this work is licensed under a creative commons attribution 4.0 international licence. licence details: http:// creativecommons.org/licenses/by/4.0/ . 1 https://sproutsocial.com/insights/social-media-statistics/ all three sub-tasks were organised for english, where as for the other languages only sub-task a was organised. our team conducted experiments for sub-tasks a and b on english language. while experimenting for sub-task a we discovered that statistical methods i.e tf-idf with logistic regression performed on par with transformer based model i.e bert. for this sub-task we achieved 22 nd rank out of 85 participants and our submission utilized an ensemble model. for sub-task b machine learning approaches like logistic regression/random forest with tf-idf, and deep leaning approaches using convolutional neural networks were not able to achieve good results. the best results were given by a generic pre-trained transformer-based language model, bert, thereby showing its prowess in categorizing offensive tweets without any explicit feature engineering. for sub-task b we achieved 11 th rank out of 43 teams who submitted their results. the rest of paper is organized as follows. in section 2 we discuss related work. in section 3 we briefly discuss about the dataset provided. proposed methods and obtained results are presented in section 4. finally we conclude our work and future works in section 5. ## related work given the pernicious nature of hate speech and offensive content on social media platforms, development of methods to mitigate their proliferation has attracted researcher from many field like computer science, computational linguistic, psychology, etc., to come forward and provide solutions. the term hate speech was first coined by @xcite . @xcite detected hate speech using word n-grams and sentiment lexicon. @xcite did an extensive study on hate speech, they identified various features for hate speech like words, sentiment, linguistic, knowledge-based features, etc. razavi et al. @xcite was the first to work on identifying offensive language using machine learning, they used rule-based methods and statistical features. recent work in offensive language detection by @xcite have focused on more fine-grained details, i.e, type and target of posts. recently organized competitions like offenseval2019 @xcite , hasoc2019 @xcite also focused on type and target of posts. in offenseval2019 most of the teams used deep learning based models like lstm, bi-lstm, cnn with pre-trained embeddings. transfer learning using pre-trained language models was a popular methodology amongst top performers, with 7 out of top 10 using bert @xcite for sub-task a. there were few teams who used machine learning algorithms like logistic regression and svm. ## dataset for the second iteration of offenseval, the organizers provided a weakly labeled dataset with huge amount of data points for training @xcite . ## conclusion in this paper we presented methods used to solve two sub-tasks from the offenseval 2020 track and our results for both tasks. for sub-task a, feature extraction using tf-idf and machine learning techniques like logistic regression/ random forest/svm, which do not capture polysemy and are unable to deal with oov words, gave results comparable to sota transfer learning technique using bert, a possible reason for this may be that the models are using the occurrence of particular words to categorize the tweets. for sub-task b methods that utilize tf-idf features did not perform as well as transfer learning using bert, probably because the task was more fine-grained and required the model to capture interactions between words in the tweet and some degree of co-reference resolution. in future we would like to improve result of sub-task b by extracting some task specific features, also do some more text-prepossessing, include emoticons as they may be used for insulting.