E nd T imes at S em E val-2021 Task 7: Detecting and Rating Humor and Offense with BERT and Ensembles (2021)

Abstract: This paper describes Humor-BERT, a set of BERT Large based models that we used in the SemEval-2021 Task 7: Detecting and Rating Humor and Offense. It presents pre and post processing techniques, variable threshold learning, meta learning and Ensemble approach to solve various sub-tasks that were part of the challenge. We also present a comparative analysis of various models we tried. Our method was ranked 4th in Humor Controversy Detection, 8th in Humor Detection, 19th in Average Offense Score prediction and 40th in Average Humor Score prediction globally. F1 score obtained for Humor classification was 0.9655 and for Controversy detection it was 0.6261. Our user name on the leader board is ThisIstheEnd and team name is EndTimes.

## introduction the purpose of this paper is to present different approaches that we tried towards various sub-tasks in semeval-2021 task 7 detecting and rating humor and offense task @xcite . it consists of following sub-tasks: we participated in all of the above sub-tasks. task 1a and task 1c are classification problems with f1 score as metric whereas, task 1b and task 2a are regression problems with rms error values as scoring criteria. all the above tasks shared the same training, development and test set @xcite . training set consisted of 8000 text sentences, containing labels for all the subtasks. size of development and test set was 1000 text sentences each. humor, is an interesting linguistic challenge. it's an abstract concept and depends a lot on audience and their interpretation of the jargon used to generate humor. notion of humor changes from culture to culture, age group, gender and social status of target audience. it is a subjective and personal phenomenon. what's humorous to one person could be non humorous or even offensive to the other. in this task, labels and ratings were collected from a balanced set of age groups from 18-70 and variety of genders, political stances and social status @xcite . humor detection and rating tasks have been the defined before but the task of detecting controversy @xcite is new and interesting. it captures the variance in humor rating due to variance in age, gender and other demographic features of the user. this task is also unique in the way that it combines humor and offense rating tasks for the same dataset. what is humorous to one user could be offensive to other depending on their demographic characteristics. ## related works humor and offense related tasks have been pursued before as well. various nlp techniques from classical n-gram techniques @xcite to transfer learning over pre-trained language models like bert @xcite have been tried for humor related tasks. @xcite extract humor anchors and use a k-nn based classifier to detect humor. @xcite used cnn and highway network whereas, (weller and seppi, 2019) used bert fine tuning to classify humor in text. apart from text, multimodal data for humor was curated by @xcite . in other works on multimodal data @xcite tag video comments data automatically and use audio data alone to predict humor. they used random forest and cnn over mfcc features. @xcite take transcripts from ted talk and detect audience laughter using cnn. ## system overview in development phase, we split the data as training:validation :: 7200:800, but for evaluation phase we take entire 8000 texts for training and 1000 size development set. later on, we also tried bagging, which will be described later. ## experiments in this section, we describe the dataset and various experiments performed. for classification tasks we used binary cross-entropy loss, whereas for regression tasks we used mean squared error as metric to be minimized. ## results and analysis we describe the results for each of the sub-tasks and analyse the results. baseline for classification task was naive bayes model with bag of words features, and for the regression task, it was support vector regression. ## conclusion and future work humor is a abstract linguistic construct. that is why a strong language model like bert-large performs really well on these tasks. our models were ranked under 10 in 2 tasks and under 20 in 1 task, this shows that bert based models outperform other models. especially, if many berts are combined using a good ensemble technique. for future work ensemble methods like stacking and blending could be tried. also, different models with different hyper parameter values could be combined in a more effective way to get better results.