G ulu at S em E val-2021 Task 7: Detecting and Rating Humor and Offense (2021)

Abstract: Humor recognition is a challenging task in natural language processing. This document presents my approaches to detect and rate humor and offense from the given text. This task includes 2 tasks: task 1 which contains 3 subtasks (1a, 1b, and 1c), and task 2. Subtask 1a and 1c can be regarded as classification problems and take ALBERT as the basic model. Subtask 1b and 2 can be viewed as regression issues and take RoBERTa as the basic model.

## introduction for social animals like humans, humor is an effective bonus. from the perspective of evolutionary psychology, "humorous" often means superior creativity, in other words, a smarter mind. therefore, it is important to recognize whether a sentence is humorous and how humorous the sentence is. it is a bit impractical to recognize such a huge data set by humans, so it becomes necessary for us to develop a system to automatically detect humor. in this task, the organizer collects labels and ratings from a balanced age group of 18-70. annotators also represent various genders, political positions, and income levels. therefore, for some texts classified as humorous, we should once again prove whether they are controversial and predict the offensiveness of the text. for more specific content, please refer to the official website of the competition foot_0 . because the pre-trained and deep learning models have shown excellent performance in many nlp problems such as classification and topic extraction @xcite , so i use deep learning methods to deal with those four tasks. according to the latest related research progress, the transformer-based language model has become my favorite model. in order to facilitate the understanding of the corresponding model of each subtask, i made it into a table shown as ## related work automatic humor recognition is a very challenging research topic in natural language processing. a person's degree of humor is largely determined by his educational knowledge and common sense of life. in addition, many types of humor require a lot of external knowledge, such as irony, metaphor and satire. @xcite first determined the semantic structure behind each structure of the humor and design feature set, and then used a calculation method to identify the humor and their humor recognizer was very effective in automatic distinction humorous and non-humorous text. @xcite proposed a generative language model based on the inconsistency theory to model humorous text, so that they can use background text sources such as wikipedia item descriptions, and can build multiple functions for recognizing humorous comments. using supervised learning to classify reviews into humorous reviews and non-humorous reviews, these functions showed that the features constructed based on the proposed generative model were more effective than the main features proposed in the existing literature. @xcite found that certain grammatical structural features are consistently related to humor. both experimental results and analysis showed that humor can be regarded as a style, and the content-independent syntactic structure can help identify humor and had good explanatory power. therefore, they proposed to use syntactic structure features to enhance humor recognition ability. compared with the baseline driven by humor theory, their method had achieved a significant improvement. and subtask 1b and 2 are regression problems. we need to predict the humor of a sentence, and because the implicit meaning of the sentence may be offensive for someone, we also need to detect the degree of attack on each sentence. traditionally, text regression is solved using linear models. @xcite proposed a method based on a deep convolutional neural network (cnn). @xcite recommended using copula: a powerful statistical framework. their model clearly outperformed a strong linear and nonlinear discrimination baseline. @xcite used cnn regression with auxiliary ordinal regression objective to predict the popularity of petitions in their work. a regression task is actually a special form of the classification task. the final output is a value rather than the probability of a specific category. therefore, the bert model can achieve good results. ## results in this task, i used albert, roberta, and bert models for the training task. for these models, the main hyperparameters i want to pay attention to are the training step size, batch size learning rate, and epoch. the parameters of my model are shown in