TECHSSN at S em E val-2021 Task 7: Humor and Offense detection and classification using C ol BERT embeddings (2021)

Abstract: This paper describes the system used for detecting humor in text. The system developed by the team TECHSSN uses binary classification techniques to classify the text. The data undergoes preprocessing and is given to ColBERT (Contextualized Late Interaction over BERT), a modification of Bidirectional Encoder Representations from Transformers (BERT). The model is re-trained and the weights are learned for the dataset. This system was developed for the task 7 of the competition, SemEval 2021.

## introduction natural language processing faces the challenges working with humor as it is a highly subjective phenomena and the age, gender and socio-economic status are known to have an impact on the perception of the joke. it usually involves multiple word senses and cultural knowledge to appreciate humor to its best. now a days chatbots and virtual assistants require automated humor detection systems for a better interaction with the user by understanding what a human-like approach to humor is. it is crucial to understand the real motive of the user and provide appropriate answer to have a better experience of the user with the virtual assistants @xcite . based on the general linguistic structure of humor, we propose an approach for detecting humor in short texts using colbert in this paper. we have developed a system in the name of techssn for previous semeval tasks @xcite for offensive language detection using various machine learning and deep learning networks. in se-meval 2021, we participated in subtask-1a and 1c of humor and offensiveness detection in task 7-hahackathon @xcite . ## related work continuous research is going on in this field of humor detection and the systems are getting better every year. de oliveira and rodrigo (2015) developed a model for humor detection in yelp reviews and used convolutional networks with a maximum of 81.57% accuracy. the system developed by ortega-bueno et al. (2018) used uo upv, a attention-based long short-term memory network. the model consists of a bidirectional lstm neural network with an attention mechanism that allows to estimate the importance of each word and then, this context vector is used with another lstm model to estimate whether the tweet is humorous or not. the f1 score for this system is approximately 0.78 with accuracy, 0.84. @xcite developed a system with bert, a multi-layer bidirectional transformer encoder which can help to learn deep bidirectional representations, and the pretrained model is fine-tuned on training data. their best f1 score on the test set is 0.784. @xcite explained the need and various methods used for offensive language detection. bert model is used with transfer learning for the offensiveness detection by @xcite with f1 score as 0.8286 and accuracy as 0.8628. ## methodology colbert base model is chosen for humor text classification which has 8 layers with the last layer's activation function as the sigmoid func-tion, as it is performing binary classification. the remaining layers have relu activation function. ## results and discussion the test dataset given by semeval organisers (gold-test-27446.csv) was tested on different models and the f1 score for all these models are discussed in the following sections.