IIITH at S em E val-2021 Task 7: Leveraging transformer-based humourous and offensive text detection architectures using lexical and hurtlex features and task adaptive pretraining (2021)

Abstract: This paper describes our approach (IIITH) for SemEval-2021 Task 5: HaHackathon: Detecting and Rating Humor and Offense. Our results focus on two major objectives: (i) Effect of task adaptive pretraining on the performance of transformer based models (ii) How does lexical and hurtlex features help in quantifying humour and offense. In this paper, we provide a detailed description of our approach along with comparisions mentioned above.

## introduction humour is an important part of human conversation. it has a social function as well and can play an important role in group cohesiveness @xcite . hence humorous content is also found on various social media websites. while there has always been a fine line between funny and offensive humour, the anonymity, distance and isolation provided by being online can increase instances of offensive or controversial humour being posted online. @xcite in this task, we have presented a transformer based approach combined with lexical and hurtlex feature sets to quantify humour and offense of a piece of text. we achieved an f1 score of 0.959 in the humor classification task and 0.592 in the humor controversy task. for the regression tasks, we achieved a rmse score of 0.541 and 0.488 in the humor regression and offense regression task respectively. ## related work there have been many attempts made at computational humour detection. in this section, we briefly describe other work in this area. in this approach @xcite , the authors have used universal language model fine-tuning method for humour recognition. convolutional neural networks (cnn) have also been used for this task by (chen and soo, 2018) whereas @xcite used transformers to classify humour. there has also been a lot of shared tasks and workshops related to computational humour. one of them is semeval-2020 task 7: assessing humor in edited news headlines @xcite where zhang @xcite used bidirectional neural networks with an attention mechanism and incorporated lexical features to assess humour in edited news headlines. there has been a lot of work done on hate speech and offensive speech detection as well. cnn's and gated recurrent units (gru) have been used for this task @xcite . recurrent neural networks combined with user-related information have also been used for hate speech detection in twitter data @xcite whereas multilingual transformer architectures were leveraged by @xcite to detect hostile content in english, hindi and german. ## task and dataset overview the task @xcite is divided into 4 sub-tasks. the dataset for the tasks was provided by the workshop organizers. it consisted of 10,000 sentences. 8,000 sentences were provided for training and 1,000 for validation. the remaining 1,000 were used for testing. each row consisted of a unique identifier,the text and the label values of "is humor", "humor rating","humor controversy" and "offense rating". ## results the gains attributed to task adaptive pretraining of roberta-base on the humour classification is shown in in task score rank task 1a (f1-score) 0.9616 14 task 1b (rmse) 0.5263 5 task 1c (f1-score) 0.6242 6 task 2 (rmse) 0.4772 23 in overall, this work shows how task adaptive pretraining can improve model performance for downstream tasks and the role of hurtlex and lexical features for humor and offensive detection. ## conclusion all the experiments performed above were done with default hyperparameters(unless explicitly mentioned) due to resource constraints. the model performances could have improved if we could search for optimal hyperparameters using cross validation. furthermore, the regression tasks could improve if we could use an ensemble of the best performing models for our final predictions.