DUTH at S em E val-2021 Task 7: Is Conventional Machine Learning for Humorous and Offensive Tasks enough in 2021? (2021)

Abstract: This paper describes the approach that was developed for SemEval 2021 Task 7 (Hahackathon: Incorporating Demographic Factors into Shared Humor Tasks) by the DUTH Team. We used and compared a variety of preprocessing techniques, vectorization methods, and numerous conventional machine learning algorithms, in order to construct classification and regression models for the given tasks. We used majority voting to combine the models’ outputs with small Neural Networks (NN) for classification tasks and their mean for regression for improving our system’s performance. While these methods proved weaker than modern, deep learning models, they are still relevant in research tasks because of their low requirements on computational power and faster training.

## introduction the underpinnings of humor have proven far more vexing than those of other emotional experiences. it is a highly subjective topic that various scholars have attempted to construct theories for understanding its fundamental elements in the studies of philosophy, linguistics, psychology and sociology. some theories, e.g. the benign violation theory @xcite , suggest that humor can be described as linguistic violations that still make grammatical sense. the aforementioned theory supports that for a joke to be classified as humorous, it needs to avoid being too harmless or too offensive. there are numerous studies in humor sentiment analysis in the last decade. in microblogging, @xcite considered extracting linguistic devices from tweets to be used as features for classifying these tweets as humorous or ironic, while @xcite approached the classification of humorous tweets as a multi-class problem of 11 types of humor, in his attempts to better attribute the real sentiment of a tweet. recent attempts on humor detection on semeval's 2020 task 7 indicate that transformer models like bert @xcite far outperform traditional machine learning algorithms @xcite . this paper describes our submissions to se-meval 2021 task 7 and is structured as follows. section 2 describes the tasks, training data, and evaluation measures. section 3 describes key methods and algorithms used. section 4 describes our proposed system while section 5 analyzes our results. finally, we draw our conclusions in section 6, where we also propose directions for future work. ## background in this section we describe each subtask's objective, the given data, and evaluation measures. ## experimental setup in this section we describe the preprocessing and vectorization methods as well as the machine learning algorithms used. ## system overview in this section we describe the proposed system for each subtask. ## results each preprocessing method had an impact on model performance, as it is shown in we can detect a pattern in the difference between the winning team's submissions and our submissions, and between the performance of our nn and non-nn models. it would be an indicationfor subtask 1a-that our conventional machine learning models, while achieving a respectable performance, cannot handle some outliers. this can be also observed through the results of subtask 2, where outliers have a greater impact on the rmse metric. our false negative results on humor are mostly ironic, reference-based jokes or highly controversial ones, and our false positive on the other hand, our systems on subtasks 1b and 1c were much closer to their superior models. while subtask's 1b results could be attributed to a large extend on the distribution of humor ratings, subtask 1c seems to be a much harder task regardless the approach. with an average accuracy of 0.5 across all submissions, humor controversiality seems to puzzle even the most complex models; anywise, humor controversiality is much more subjective than humor itself. nevertheless, a great advantage of conventional machine learning is training speed and hardware requirements. state-of-the-art boosting models like the ones we used (lightgbm and xgb) can be accelerated through the use of gpus while our small nns can be trained in a couple of minutes. training/tuning deep learning models, on the other hand, requires expensive hardware and can be very time-consuming. ## conclusions in this report, we presented our approach on humorous and offensive text classification and rating based on the combination of outputs from different preprocessing techniques, vectorization methods and machine learning algorithms. our proposed systems were outperformed by other teams in the main tasks, while our conventional machine learning models were mostly inferior to our neural networks. our future work will focus on expanding our preprocessing methods, introducing further ensemble methods and stacking, as well as