Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognition (2021)

Abstract: Humor recognition has been widely studied as a text classification problem using data-driven approaches. However, most existing work does not examine the actual joke mechanism to understand humor. We break down any joke into two distinct components: the set-up and the punchline, and further explore the special relationship between them. Inspired by the incongruity theory of humor, we model the set-up as the part developing semantic uncertainty, and the punchline disrupting audience expectations. With increasingly powerful language models, we were able to feed the set-up along with the punchline into the GPT-2 language model, and calculate the uncertainty and surprisal values of the jokes. By conducting experiments on the SemEval 2021 Task 7 dataset, we found that these two features have better capabilities of telling jokes from non-jokes, compared with existing baselines.

## introduction one of the important aspects of computational humor is to develop computer programs capable of recognizing humor in text. early work on humor recognition @xcite proposed heuristic-based humor-specific stylistic features, for example alliteration, antonymy, and adult slang. more recent work @xcite @xcite regarded the problem as a text classification task, and adopted statistical machine learning methods and neural networks to train models on humor datasets. however, only few of the deep learning methods have tried to establish a connection between humor recognition and humor theories. thus, one research direction in humor recognition is to bridge the disciplines of linguistics and artificial intelligence. in this paper, we restrict the subject of investigation to jokes, one of the most common humor types in text form. as shown in figure 1 , these jokes usually consist of a set-up and a punchline. the set-up creates a situation that introduces the hearer into the story framework, and the punchline concludes the joke in a succinct way, intended to make the hearer laugh. perhaps the most suitable humor theory for explaining such humor phenomenon is the incongruity theory, which states that the cause of laughter is the perception of something incongruous (the punchline) that violates the hearer's expectation (the set-up). based on the incongruity theory, we propose two features for humor recognition, by calculating the degree of incongruity between the set-up and the punchline. recently popular pre-trained language models enable us to study such relationship based on large-scale corpora. specifically, we fed the set-up along with the punchline into the gpt-2 language model @xcite , and obtained the surprisal and uncertainty values of the joke, indicating how surprising it is for the model to generate the punchline, and the uncertainty while generating it. we conducted experiments on a manually labeled humor dataset, and the results showed that these two features could better distinguish jokes from non-jokes, compared with existing baselines. our work made an attempt to bridge humor theories and humor recognition by applying large-scale pre-trained language models, and we hope it could inspire future research in computational humor. ## related work humor data @xcite created a one-liner dataset with humorous examples extracted from webpages with humor theme and non-humorous examples from reuters titles, british national corpus (bnc) sentences, and english proverbs. @xcite scraped puns from the pun of the day website 1 and negative examples from various news websites. there is also work on the curation of non-english humor datasets @xcite . @xcite developed ur-funny, a multimodal humor dataset that involves text, audio and video information extracted from ted talks. humor recognition most of the existing work on humor recognition in text focuses on one-liners, one type of jokes that delivers the laughter in a single line. the methodologies typically fall into two categories: feature engineering and deep learning. @xcite designed three human-centric features (alliteration, antonymy and synonym) for recognizing humor in the curated oneliner dataset. @xcite approached the problem by calculating the semantic relatedness between the set-up and the punchline (they evaluated 150 one-liners by manually splitting them into "setup" and "punchline"). @xcite investigated funny captions for cartoons and proposed several features including perplexity to distinguish between funny and less funny captions. @xcite proposed a probabilistic model and leveraged background text sources (such as wikipedia) to identify humorous yelp reviews. @xcite proposed to model sentiment association between elementary discourse units and designed features based on discourse relations. @xcite explored the usage of word associations as a semantic relatedness feature in a binary humor classification task. with neural networks being popular in recent years, some deep learning structures have been developed for the recognition of humor in text. chen and lee (2017) and 1 http://www.punoftheday.com/ chen and soo (2018) adopted convolutional neural networks, while @xcite used a transformer architecture to do the classification task. @xcite incorporated extra phonetic and semantic (ambiguity) information into the deep learning framework. in addition to these methodological papers, there are also some tasks dedicated to computational humor in recent years. semeval 2020 task 7 @xcite aims at assessing humor in edited news headlines. semeval 2021 task 7 @xcite involves predicting the humor rating of the given text, and if the rating is controversial or not. in this task, @xcite adopted the deberta architecture @xcite with disentangled attention mechanism to predict the humor labels. although the work of @xcite is the closest to ours, we are the first to bridge the incongruity theory of humor and large-scale pretrained language models. other work @xcite has attempted to predict punchlines in conversations extracted from tv series, but their subject of investigation should be inherently different from ours-punchlines in conversations largely depend on the preceding utterances, while jokes are much more succinct and self-contained. ## humor theories the attempts to explain humor date back to the age of ancient greece, where philosophers like plato and aristotle regarded the enjoyment of comedy as a form of scorn, and held critical opinions towards laughter. these philosophical comments on humor were summarized as the superiority theory, which states that laughter expresses a feeling of superiority over other people's misfortunes or shortcomings. starting from the 18 th century, two other humor theories began to challenge the dominance of the superiority theory: the relief theory and the incongruity theory. the relief theory argues that laughter serves to facilitate the relief of pressure for the nervous system @xcite . this explains why laughter is caused when people recognize taboo subjects-one typical example is the wide usage of sexual terms in jokes. the incongruity theory, supported by kant (1790), schopenhauer (1883), and many later philosophers and psychologists, states that laughter comes from the perception of something incongruous that violates the expectations. this view of humor fits well the types of jokes commonly found in stand-up comedies, where the set-up establishes an expectation, and then the punchline violates it. as an expansion of the incongruity theory, raskin (1979) proposed the semantic script-based theory of humor (ssth) by applying the semantic script theory. it posits that, in order to produce verbal humor, two requirements should be fulfilled: (1) the text is compatible with two different scripts; (2) the two scripts with which the text is compatible are opposite. ## methodology the incongruity theory attributes humor to the violation of expectation. this means the punchline delivers the incongruity that turns over the expectation established by the set-up, making it possible to interpret the set-up in a completely different way. with neural networks blooming in recent years, pretrained language models make it possible to study such relationship between the set-up and the punchline based on large-scale corpora. given the set-up, language models are capable of writing expected continuations, enabling us to measure the degree of incongruity, by comparing the actual punchline with what the language model is likely to generate. in this paper, we leverage the gpt-2 language model @xcite , a transformer-based architecture trained on the webtext dataset. we chose gpt-2 because: (1) gpt-2 is already pretrained on massive data and publicly available online, which spares us the training process; (2) it is domain independent, thus suitable for modeling various styles of english text. our goal is to model the set-up and the punchline as a whole piece of text using gpt-2, and analyze the probability of generating the punchline given the set-up. in the following text, we denote the set-up as x, and the punchline as y. basically, we are interested in two quantities regarding the probability distribution p(y|x): uncertainty and surprisal, which are elaborated in the next two sections. ## visualizing uncertainty and surprisal to get a straightforward vision of the uncertainty and surprisal values for jokes versus non-jokes, we plot their histograms in figure 3 (for all 3,052 labeled examples). it can be observed that, for both uncertainty and surprisal, jokes tend to have higher values than non-jokes, which is consistent with our expectations in section 4. ## conclusion this paper makes an attempt in establishing a connection between the humor theories and the nowadays popular pre-trained language models. we proposed two features according to the incongruity theory of humor: uncertainty and surprisal. we conducted experiments on a humor dataset, and the results suggest that our approach has an advantage in humor recognition over the baselines. the proposed features can also provide insight for the task of two-line joke generation-when designing the text generation algorithm, one could exert extra constraints so that the set-up is chosen to be compatible with multiple possible interpretations, and the punchline should be surprising in a way that violates the most obvious interpretation. we hope our work could inspire future research in the community of computational humor.