A Survey on Approaches to Computational Humor Generation (2020)

Abstract: We provide a comprehensive overview of existing systems for the computational generation of verbal humor in the form of jokes and short humorous texts. Considering linguistic humor theories, we analyze the systematic strengths and drawbacks of the different approaches. In addition, we show how the systems have been evaluated so far and propose two evaluation criteria: humorousness and complexity. From our analysis of the field, we conclude new directions for the advancement of computational humor generation.

## introduction since the early 1990s, computer scientists and linguists have been trying to create jokes using algorithms. despite this longstanding research and the current evolution of powerful approaches in deep learning and neural networks -which are very well able to simulate human behavior and creativity, e.g. with the composition of pop music @xcite , poetry @xcite or abstract paintings @xcite -no system has been able to imitate human humor and to generate jokes like a human until today. from a research perspective, the task at hand is called humor generation. it is part of the broader field of computational humor and the output of humor generation systems are typically, but not exclusively, jokes. besides the creation of humor, the following aspects are relevant for the field of computational humor as a whole: while humor generation can be researched from both the textual and the visual perspective @xcite , this survey article is dedicated to providing an overview of only text-based systems and approaches for the computational generation of humor. although only a minority of the existing systems refer to humor theory explicitly, we assume that it is necessary to also consider some of the most current linguistic humor theories when discussing different computational approaches to humor generation. among the most prominent humor theories are the following: although research on computational humor generation has been going on since the early 1990s, the field is still very heterogenous. research is conducted by at least two different communities: humor theory/linguistics (e.g. @xcite @xcite and natural language processing/computational linguistics (e.g. @xcite @xcite . there have been attempts by the humor theory community to streamline the research activity in a series of workshops @xcite , however, the vast majority of existing humor systems originates from the nlp community and does not seem to pick up those more theory-driven impulses. interestingly enough, most of the older nlp approaches also do not take into account related and previous work and for the most part seem to exist in isolation from each other @xcite . after all, a more recent approach aims to standardize terminology and generalize the computational templatebased stream of research @xcite . this framework is, however, not able to capture the entirety of the existing systems and the only existing survey paper that aims at capturing activity in the field dates back to 2001 @xcite . with the present survey paper, we try to fill that gap by providing a comprehensive overview of the existing approaches. to be able to compare and discuss these approaches, we propose humorousness and complexity as the two main criteria for the evaluation of humor generation systems. by analyzing the strengths and weaknesses of the respective systems with regard to those criteria, we conclude directions for future research in the field. ## humor generation systems humor generation can be viewed as a special case of automatic text generation. accordingly, the existing systems can be categorized as belonging to one of two major approaches to text generation: templates and neural networks. until yang and sheng's publication on an lstm rnn for joke production in 2017, all of the previous published systems were template-based. while the early template-based systems had to be equipped with handcrafted lexicons, more recent systems have a variety of external lexical resources as source of material for generation. it is also worth mentioning that the nature of the humorous texts produced by different systems is rather diverse. while a number of systems was focused on the generation of question-answer jokes @xcite @xcite @xcite , others aimed at creating narrative jokes @xcite @xcite . furthermore, three systems generate humor through lexical replacement, in acronyms @xcite , proverbs @xcite or in sms @xcite , and one system creates witty analogies @xcite . in the following, we will give an overview of existing humor generation systems. since neural networks have recently achieved state-of-the-art results on many tasks in natural language processing, we will start with the group of systems that apply them for the creation of jokes. this group, however, contains only two systems, whereas the vast majority (ten systems) belongs to the group of templatebased approaches. ## evaluation of humor generation systems in order to compare the existing systems to each other, some basic measures for the quality of humor generation systems are needed. although a multitude of systems has been developed so far, a standardized methodology of assessing the quality of such systems is still missing. most previous systems chose the humorousness of the output as an evaluation criterion. the majority of approaches was evaluated by means of user studies where actual humans rated the generated jokes on a likert scale that typically ranges from 0 (not a joke) to 5 (really funny). the first authors to introduce this method were binsted and ritchie (1994) and many followed their example, such as @xcite or @xcite . @xcite as well as @xcite employ a four-step likert scale. however, in most of the papers it remains unclear how the samples for evaluation were selected. petrovic and matthews (2013) for example disclose that they hand-picked the best jokes for the human assessment. obviously, it matters a lot whether the average humor of a system is calculated on the basis of random jokes or on the basis of jokes that were pre-selected by the authors. the main challenge in the design of humor evaluation measures is that humor is highly subjective and what makes one person laugh might not have the same effect on another person. @xcite show that the funniness judgements of jokes differ significantly among the test persons. circumventing this issue, @xcite proposes the hf (humorous frequency) as measure for the evaluation of computational humor generators. he defines hf as the fraction of funny items in a randomly generated set of jokes. any item is either funny or not funny, i.e. in this context humorousness has a boolean value rather than a score. to calculate the hf of a system, boolean humorousness ratings have to be assigned to every item of a randomly selected set of jokes generated by the system. however, how a single joke is assessed remains highly subjective. @xcite suggest to classify a computer-generated joke as successful when people cannot tell it apart from a human-generated joke, funniness left aside. @xcite conduct such an evaluation. the authors design what they call a soft turing test to let test persons rate the humanness of the jokes generated by their system on a three-score likert-scale (2-definitely human/0-definitely machine). what all these approaches have in common, however, is that they do not consider the humor capability of the systems as a whole and rather evaluate individual jokes, or, as in the case of the hf, derive an overall evaluation of the system from the scores of the individual jokes. consequently, the diversity of the produced material is completely disregarded in these evaluations. this overlooks the fact that a system that produces a row of almost similar jokes would not be very enjoyable for a human. taking this factor into account, the humor capability of a system with a low complexity, i.e. a system that produces similar jokes with almost the same wording and the same production principle, should be rated lower than a system that produces a variety of jokes and joke mechanisms. to our knowledge, none of the evaluation procedures yet has attempted to apply objective criteria to the decision of whether a given piece of text is humorous or not. however, from the insights of humor theory and research, criteria for what constitutes a joke can be concluded. consequently, a useful measure would be not if a text is humorous or how funny a joke is, but rather whether a text is identifiable as joke or not, taking into account objective criteria. ## comparison of humor generation systems based on their humorousness and complexity the considerations in the last section give rise to two main criteria for the evaluation of humor systems: humorousness and complexity. in the following, we provide a discussion of the benefits and limitations of different systems based on these criteria. we define humorousness as the ability of a humor system to output text that is identifiable as a joke or a text with a humorous intent. the above introduced incongruity theories can contribute objective criteria towards this end. in this sense, a text is humorous if it conveys two different concepts, whose incompatibility is resolved in an unexpected way. we define complexity as the ability of a humor system to generate a variety of humorous texts, concerning syntactical and lexical features, as well as the joking mechanism. the joking mechanism describes the way in which the incongruity and its resolution is evoked. based on these two criteria, with regard to complexity, we assigned rank 3 to systems without template that restricts syntactical and lexical features, rank 2 to systems that have either a variety of different templates or template with variable slots that are large enough to allow a lexical diversity and rank 1 to systems that have only one template. this analysis reveals the strengths and weaknesses of the two approaches to humor generation. the table clearly shows that none of the systems reaches the highest rank in humorousness as well as in complexity. the template-based approaches fill the higher ranks for humorousness by implicitly including humor theory through their constraints and metrics. the neural systems on the other hands are not capable of generating humorous text at all, scoring the lowest ranks. nonetheless, these systems reach the highest scores in complexity through their learned language model, consequently being able to generate a large variety of text. the template-based approaches fail to reach that level of complexity. ## conclusion: toward better humor generation systems by means of linguistic humor theories our survey shows that even after more than 25 years of work on humor generation, further research needs to be conducted in order to create a system that generates text that fulfils theory-driven criteria for humor and that is able to produce a variety of texts, regarding syntactical structures and joke mechanisms. it is somewhat surprising that almost none of the present approaches refers to a linguistic humor theory explicitly. the initially introduced humor theories and their evolution, however, make very clear that textual humor is evoked by complex semantic mechanisms. a number of requirements about the involved concepts and their semantic relationships needs to be fulfilled in order to create a text that is actually humorous rather than just puzzling. these theories enable us to explain why some texts are funny and make people laugh and others do not. hence, we believe that considering these existing humor theories will be an important step to overcome the limitations of existing joke generators. we regard the surprise ambiguation model and the ssth as the most interesting theories. the surprise ambiguation model, as formalized by @xcite , defines a small set of properties and entities. based on the three entities m1 -the obvious interpretation of the set-up, m2 -the hidden interpretation of the set-up and m3 -the meaning of the punchline, the model defines the following relations and properties involved in humorous texts: obviousness (m1 is more likely than m2 to be noticed by the reader), conflict (m3 does not make sense with m1), compatibility (m3 does not make sense with m2), comparison (there is some contrastive relationship between m1 and m2) and inappropriateness (m2 is inherently odd or a taboo). @xcite already used these properties to identify and validate a set of metrics that are used by machine learning algorithms to extract features from existing jokes. also, the concept of humor postulated by the ssth has very few requirements that could be formalized pretty easily. the ssth even implicitly gives an instruction of how a joke can be constructed from non-humorous content. in suls' two-stages model on the other hand, whether something is humorous or not is dependent on the presence of a cognitive rule. however, a cognitive rule can be anything and is therefore not very feasible for being modeled computationally. the gvth focuses more on a representation of the differences of jokes rather than on the humor mechanism itself. we see great potential for the improvement of existing approaches to computational humor generation by taking into account these theoretical approaches to humor and hope that this survey will pave the way for further, more theory-driven research in this direction.