A Unified Framework for Pun Generation with Humor Principles (2022)

Abstract: We propose a unified framework to generate both homophonic and homographic puns to resolve the split-up in existing works. Specifically, we incorporate three linguistic attributes of puns to the language models: ambiguity, distinctiveness, and surprise. Our framework consists of three parts: 1) a context words/phrases selector to promote the aforementioned attributes, 2) a generation model trained on non-pun sentences to incorporate the context words/phrases into the generation output, and 3) a label predictor that learns the structure of puns which is used to steer the generation model at inference time. Evaluation results on both pun types demonstrate the efficacy of our model over strong baselines.

## introduction recently, computational humor theories investigating why puns are funny have shown high correlations with human judgments. @xcite use a probabilistic model to decompose puns into two dimensions: ambiguity of meaning and distinctiveness of viewpoints, and show that these two aspects combined have the strongest alignment with human judgments (@xmath0. @xcite show that ambiguity/distincitveness alone cannot capture the whole picture, and develop an additional metric to measure how much surprise is aroused when the pun word and alternative word are flipped. for example in figure 1 , the pun word is soled and the alternative word is sold. seeing soled in the phrase 'were soled at the store at half price' instead of sold arouses surprise in the local context but makes sense in the global context. despite the success in identifying important linguistic traits of successful pun sentences, how to incorporate these aforementioned theories into the pair (e.g. 'soled-sold') is the input. after retrieving a suitable context word and a phrase, we use a pun label predictor to steer the base gpt-2 model to produce puns. labels d1/d2/a mean the next word should be distinct to (supporting) the pun word, distinct to (supporting) the alternative word, or ambiguous. a '-' mark means the label predictor is less confident and thus we do not intervene the generation process. although @xcite propose a retrieve-andedit approach to incorporate surprise, their error analysis shows that the proposed retrieval methods are often unsuccessful. moreover, existing works on pun generation are split up in terms of generating homographic puns, wherein the same written word has two or more meanings @xcite @xcite , and homophonic puns, where two words that sound similar have different meanings @xcite @xcite . there lacks a unified generation framework for both types of puns. in this work, we incorporate all three principles: ambiguity, distinctiveness, and surprise into pun generation, and bridge the gap between the two pun types. we hypothesize that there is a learnable structure for puns regardless of the pun type, and propose a unified framework by converting homographic puns to homophonic ones. specifically, we carefully extract from a non-pun corpus 1) a context word that supports the meaning of the pun word, and 2) a phrase that is both characteristic to the alternative word and compatible with the pun word. next, we train a discriminator on existing homophonic pun data to learn the structure of a pun -the type of each word in the sentence, which could be one of 'a' -ambiguous, 'd1' -distinct to the pun word, or 'd2' -distinct to the alternative word. one challenge, however, is that there are no ground truth labels. to this end, we collect a small amount of human annotations and boost from weak, unsupervised models to stronger, supervised models. at inference time, a label predictor is used to guide a base gpt-2 model to generate puns. at each generation step, we re-score the tokens generated by the base language model according to the predicted type, except for the case when the label predictor's confidence is under a set threshold. our model outperforms existing baselines for both pun types. pun generation. existing works on pun generation often rely on naive intuitions of semantic ambivalence. for example, @xcite and @xcite promote the ambivalence of the pun word via a constrained language model and reinforcement learning; others find related words to support semantic ambiguity @xcite . however, these systems lack serious theoretical backbones and therefore none could evaluate their generated results with regards to the proposed intuitions. what's more, the nature of 'ambivalence' alone leads to generic/boring word choice and short outputs. by incorporating distinctiveness and surprise, we ensure that the generated puns are informative and interesting. one reason that previous works leverage those simple intuitions to generate puns @xcite @xcite is that the small corpora size @xcite makes it impractical to train generation models end-to-end using human written puns. we hence propose to learn the structure of puns instead of the actual texts, which requires far less data to train on. finally, all previous works (except a concurrent one @xcite ) can only generate either homographic puns or homophonic puns. leveraging the shared structure of puns regardless of the pun type, our model can generate both pun types. ## methodology the input to our system is a pun word-alternative word pair (pw-aw, e.g., soled-sold), and the target output is a high-quality pun sentence that contains pw, e.g., 'the leather boots he was wearing were heavily abraded, and were soled at the store at half price.' in this section, we first describe the three components to generate homophonic puns as shown in figure 1 : a context word and phrase selector, a label predictor and the procedure of curating training signals, and the generation module in section 3.1 to 3.3. then, we migrate the whole system to homographic puns in section 3.4. ## conclusion we propose a novel pun generation approach that incorporates three humor principles. to this end, we learn the sentence structures from humanwritten puns, and convert the task of homographic pun generation to homophonic pun generation. our model achieves strong performance for both types.