M agic P ai at S em E val-2021 Task 7: Method for Detecting and Rating Humor Based on Multi-Task Adversarial Training (2021)

Abstract: This paper describes MagicPai’s system for SemEval 2021 Task 7, HaHackathon: Detecting and Rating Humor and Offense. This task aims to detect whether the text is humorous and how humorous it is. There are four subtasks in the competition. In this paper, we mainly present our solution, a multi-task learning model based on adversarial examples, for task 1a and 1b. More specifically, we first vectorize the cleaned dataset and add the perturbation to obtain more robust embedding representations. We then correct the loss via the confidence level. Finally, we perform interactive joint learning on multiple tasks to capture the relationship between whether the text is humorous and how humorous it is. The final result shows the effectiveness of our system.

## introduction humor is the tendency of experiences to provoke laughter and provide amusement. regardless of gender, age or cultural background, it is a special way of language expression to provide an active atmosphere or resolve embarrassment in life while being an important medium for maintaining mental health @xcite . recently, with the rapid development of artificial intelligence, it becomes one of the most hot research topics in natural language processing to recognize humor @xcite . the task of humor recognition consists of two subtasks: whether the text contains humorous and what level of the humor it is. early humor recognition methods tackle this task mainly by designing heuristic humor-specific features on classification models @xcite and have proved that this automatic way can attain satisfactory performance. nowadays, researchers try to resolve this task by statistical machine learning or deep learning technologies. the semeval 2021 task 7, hahackathon: detecting and rating humor and offense, consists of four subtasks: subtask 1 simulates the previous humor detection task, in which all scores are averaged to provide an average classification score. subtask 1a is a binary classification task to detect whether the text is humorous. subtask 1b is a regression task to predict how humorous it is for ordinary users in a value range from 0 to 5. subtask 1c is also a binary classification task to predict whether the humor grade causes controversy if the text is classified as humorous. subtask 2 aims to predict how offensive text for an ordinary user is in an integral value range from 0 and 5. due to the highly subjective nature of humor detection, the data is labeled by people with different profile in gender, age group, political position, income level, social status, etc. the tasks are extremely challenging because they lack a unified standard to define humor. to tackle the tasks, we first preprocess the text, including stemming, acronym reduction, etc. we then apply the pre-trained language model to get the representation of each subword in the text as the model input. meanwhile, we add a perturbation to the embedding layer and design an optimization goal that maximizes the perturbation of the loss function. after that, we perform interactive multitask learning on judging whether humor exists and predicting how humorous it is. that is, based on maximizing the likelihood estimation under the gaussian distribution with the same variance, we construct a multi-task loss function and automatically select different loss weights in the learning to improve the accuracy of each task. ## related work the early stages of humor recognition are based on statistical machine learning methods. for example, @xcite try to learn statistical patterns of text in n-grams and provide a heuristic focus for a location of where wordplay may or may not occur. @xcite show that automatic classification techniques can be effectively deploy to distinguish between humorous and non-humorous texts and obtain significant improvement over the apriori algorithm, a wellknown baseline. in addition, three human-centric features are designed for recognizing humor in the curated one-liner dataset. @xcite apply svm models for humor recognition as a binary classification task and prove that the technique of metaphorical mapping can be generalized to identify other types of double entendre and other forms of humor. @xcite present several modifications of the original recurrent neural network language model to solve the humor recognition task. @xcite collect a crowdsourced corpus for humor classification from spanish tweets and conduct extensive experiments to compare various machine learning models, such as support vector machine (svm), a multinomial version of naïve bayes (mnb), decision trees (dt), k nearest neighbors (knn), and a gaussian version of naïve bayes (gnb). @xcite observe that bigram language models performed slightly better than trigram models and there is some evidence that neural network models can outperform standard back-off n-gram models. chen and soo (2018) extend the techniques of automatic humor recognition to different types of humor as well as different languages in both english and chinese and proposed a deep learning cnn architecture with high way networks that can learn to distinguish between humorous and nonhumorous texts based on a large scale of balanced positive and negative dataset. with the rapid development of deep learning technology, various pre-training models have made great progress in the field of natural language processing @xcite @xcite . @xcite propose to model sentiment association between elementary discourse units and compare various cnn methods of humor recognition. @xcite employ a transformer architecture for its advantages in learning from sentence context and demonstrate the effectiveness of this approach and show results that are comparable to human performance. @xcite propose a new algorithm enhance-ment inference bert (ei-bert) that performs well in sentence classification. @xcite propose an internal and external attention neural network (ieann) attention mechanism @xcite has been applied and show good model performance. the existing work can be borrowed or inspired our proposal in this paper. ## overview in the following, we present the implementation of our system for the competition. ## experiments in the following, we present the data, experimental setup and analyze the results. ## conclusion and future work this paper presents our system for semeval-2021 task 7. several techniques, such as interactive multi-task joint training, adversarial training, and loss correction, are applied to tackle the task. more specifically, the perturbation is first added to the input embedding layer and the predicted labels are also added with the real labels to reduce the loss of the noise point data. next, the output of task 1a by the softmax is concatenated with the input of the task 1b to perform joint training on both tasks. meanwhile, the uncertainty weighting scheme on the loss allows the simple task to have a higher weight. finally, multiple models are ensembled to yield the final prediction results. our system attains the first place in the competition. in the future, we can explore and verify three other effective strategies. the first strategy is the task-adaptive funetuning on the pre-trained language models. relevant sentences can be continuously fed into the pre-trained language models to improve the model performance. the second strategy is to build a graph neural network (gnn) model to exploit all vocabulary for text classification. because bert is relatively limited to capture the global information from a larger language vocabulary, it is promising to facilitate the gnn, which captures the global information, with the indepth interaction of bert's middle layers, which embed sufficient local information. we will further investigate discourse structures @xcite for humor detection. because, both bert and gnn models information from word relations, it is necessary to involve the study of discourse structures, which describe how two sentences are