Transfer Learning for Humor Detection by Twin Masked Yellow M uppets (2022)

Abstract: Humorous texts can be of different forms such as punchlines, puns, or funny stories. Existing humor classification systems have been dealing with such diverse forms by treating them independently. In this paper, we argue that different forms of humor share a common background either in terms of vocabulary or constructs. As a consequence, it is likely that classification performance can be improved by jointly tackling different humor types. Hence, we design a shared-private multitask architecture following a transfer learning paradigm and perform experiments over four gold standard datasets. Empirical results steadily confirm our hypothesis by demonstrating statistically-significant improvements over baselines and accounting for new state-of-the-art figures for two datasets.

## introduction humor has been studied in fields such as psychology @xcite and linguistics @xcite . in natural language processing, the tasks of humor classification @xcite @xcite and generation @xcite have recently gained importance although they have been subject of reflection for some time @xcite 1 . humor can be expressed in different forms (examples in most related works on humor classification have treated the different forms of humor independently. here, we hypothesize that different forms of humor are closely related, both in terms of vocabulary 1 some efforts have recently tackled multimodal information @xcite . ## related work initial attempts have been proposed by @xcite , where humor-specific stylistic features and content-based features are combined to classify short sentences. @xcite compute acoustic-prosodic features, such as pitch and energy, in addition to the linguistic features within spoken conversations. @xcite tackle humor recognition in tweets based on phonetic, morpho-syntactic, lexicosemantic, pragmatic and affective features. @xcite combine hierarchical continuous representations with high-level features (e.g. structural features, antonyms, sentiment) to predict humor of body-punchlines in tv-sitcoms dialogues. @xcite propose a convolutional neural network (cnn)-based architecture combined with highway networks @xcite . @xcite propose a new task, which consists in recognizing whether a joke is funny or not. for that purpose, they build the reddit dataset and design a straightforward bert architecture, which competes with human perception. further experiments on puns and shortjokes, show that contextualized embeddings are strong representations for humour recognition, also upgrading (chen and soo, 2018) results. @xcite design a multilingual model based on a pre-trained (chinese, russian, spanish) bert, that is fine-tuned on intersentence relationship and sentence discrepancy prediction for body-punchlines. similar works are proposed by @xcite to recognize humorous questions in product q&a systems, and @xcite , who formalize uncertainty and surprise for body-punchlines in english. ## shared-private multitask architecture in order to take advantage of the different humor types, we propose a shared-private multitask architecture @xcite . the model depicted in figure 1 consists of a frozen shared bert @xcite layer, which is pre-trained on two different tasks to account for different humor types, and a private bert layer, which is fine-tuned on each dataset independently. ## experimental setups all models have been implemented using pytorch @xcite and hugginface @xcite libraries. all models are based on bert base foot_0 . the embedding size d for h cls is 768. for training bert with the mlm objective, each word is masked with a probability of 0.15, and we use a batch size of 6 and a learning rate of 2 × 10 -5 . for training on the humor classification task, for both the shared bert and shared-private architecture, we use a batch size of 16 and a learning rate of 2 × 10 -5 . we use the adam optimizer with a default weight decay of 0.01. for each dataset, the model is trained for 4 epochs. the best model is saved based on the development set accuracy results. code and datasets are available at https://github.com/ aseemarora1995/humor-detection . ## results analysis experimental results are illustrated in in as explained in the §3.2, the shared bert is pretrained for humor classification using a balanced shared dataset, to explain the importance of using a balanced dataset, we perform experiments by pre-training the shared bert on a complete training sets combined from all the four datasets, without taking care of balance between humor types. results are shown in the ## error analysis in ## conclusion humor is an important part of human communication. in this paper, we hypothesize that different forms of humor share a common background, and as a consequence, additional usage of one form can help in better understanding other forms in humor classification. so, we propose a shared-private multitask architecture that achieves new state-of-theart performances for two out of four datasets, and evidences strong robustness. this latter issue is crucial for humorous text generation @xcite . nevertheless, we observe that current models still have limited capacity to understand such complicated forms of humor where polysemy, external knowledge, context, and reasoning are important.