There Once Was a Really Bad Poet, It Was Automated but You Didn’t Know It (2021)

Abstract: Limerick generation exemplifies some of the most difficult challenges faced in poetry generation, as the poems must tell a story in only five lines, with constraints on rhyme, stress, and meter. To address these challenges, we introduce LimGen, a novel and fully automated system for limerick generation that outperforms state-of-the-art neural network-based poetry models, as well as prior rule-based poetry models. LimGen consists of three important pieces: the Adaptive Multi-Templated Constraint algorithm that constrains our search to the space of realistic poems, the Multi-Templated Beam Search algorithm which searches efficiently through the space, and the probabilistic Storyline algorithm that provides coherent storylines related to a user-provided prompt word. The resulting limericks satisfy poetic constraints and have thematically coherent storylines, which are sometimes even funny (when we are lucky).

## introduction a limerick is a short and catchy 5-line poem that tells a funny, crude, or ironic story. it has strict structural constraints such as an aabba rhyming scheme, a 99669 syllable count, and an anapestic meter pattern @xcite . writing limericks is a challenging task even for human poets, who have to carefully choose, optimize, and even invent new words to satisfy all of the constraints while incorporating creativity and humor. prior to this paper, there has not been a successful attempt at realistic automatic limerick generation. perhaps this is because the task is challenging: large-scale neural networks often fail to generate decent limericks because the amount of available human-written limericks to learn from is much smaller than other forms of poetry, and because limericks must follow strict structural, meter, and rhyming constraints. traditional methods for generating limericks instead hard-code the constraints into a template, so that the constraints are obeyed but the generated poems are all extremely similar (resembling mad libs, where one fills words into a single template). in this paper, we introduce a novel system of algorithms for automatic limerick generation, denoted as limgen. limgen takes a user-specified prompt word and produces a creative and diverse set of limericks related to the prompt. limgen is a rule-based search method. its main components are: (1) adaptive multi-templated constraints (amtc), which constrain limgen's search to a space of realistic limericks, leveraging knowledge from limerick sentence structures extracted from human poets; (2) the novel multi-templated beam search (mtbs), which searches the space in a way that fosters diversity in generated poems; and (3) the probabilistic storyline algorithm, which provides coherent storylines that are thematically related to the prompt word. limgen relies on the part-of-speech (pos) limerick templates extracted from a small training set and uses a pre-trained language model to fill words into the templates. we used the 345m version of pre-trained gpt-2 @xcite , which performs extremely well in unconstrained text generation. however, it is important to note that a language model such as gpt-2, powerful though it may be, is only a plugin module for lim-gen. without limgen, gpt-2 alone is completely incapable of generating limericks. through our experiments, we demonstrate that limgen creates a new benchmark for limerick generation, outperforming both traditional rulebased algorithms and encoder-decoder style neural networks across a variety of metrics, including (a) prompt: ''money'' there was a greedy man named todd, who lost all his money in a fraud. when he returned to work, he was robbed by his clerk, and never could buy a cod. (b) prompt: ''cunning'' there was a magician named nick, who fooled all his family in a trick. when he returned to hide, he was found by his bride, and killed with a magical lipstick. the main contributions of this paper are the multi-template-guided limgen system and its mtbs search algorithm. equipped with amtc, limgen is the first fully automated limerick generation system that has the ability to write creative and diverse limericks, outperforming existing state-of-the-art methods. our diversity-fostering beam search (mtbs) is on par with some of the best beam search algorithms in terms of its ability to optimize limerick quality, and it does a significantly better job at fostering diversity than other methods. the code for limgen as well as the complete list of machine-generated limericks used in our experiments are available online @xcite . from a broader perspective, we have shown that rule-based poetry generation systems that follow a multi-templated approach, as implemented via the amtc in this work, can perform better than largescale neural network systems, particularly when the available training data are scarce. our work indicates that a computational system can exhibit (what appears to be) creativity using domainspecific knowledge learned from limited samples (in our case, pos templates extracted from human-written poems). although we only use templates to capture the part-of-speech structure of limericks, in general, templates can represent any explicit or latent structures that we wish to leverage. other nlp applications (e.g., biography generation, machine translation, image captioning, machine translation) have also seen revived interest in template-guided approaches @xcite @xcite . thus, it is conceivable that the general framework of limgen, including amtc and the mtbs algorithm, can be applied to other forms of poetry generation, as well as broader domains in nlp. ## related literature to the best of our knowledge, poevolve @xcite , which combines an rnn with an evolutionary algorithm, and the stochastic hill climbing algorithm proposed by @xcite are the only other serious attempts at limerick generation in the past 20 years. unfortunately, their implementations did not result in readable limericks, as can be seen in section 4.3. traditional rule-based methods in poetry generation are able to enforce hard constraints, such as rhyming dictionaries or pos templates @xcite @xcite . limgen is also rule-based, though it has substantially more flexibility and diversity than @xcite approach, which follows a single pos template during poetry generation. needless to say, the use of adaptive multi-templates makes amtc the bedrock of limgen. neural language models have recently been able to produce free-style (unconstrained) english poetry with moderate success @xcite . in chinese poetry generation @xcite @xcite , research has been so successful that it has spurred further efforts in related areas such as sentiment and stylecontrollable chinese quatrain generation @xcite @xcite . however, their large-scale neural network models take advantage of the chinese quatrain database, which has more than 150k training examples. in contrast, limgen uses less than 300 limericks. most modern poetry-generation systems are encoder-decoder style recurrent networks (e.g., character-level and word-level lstms) with modifications such as various forms of attention mechanisms. lau et al. (2018) integrated these techniques and proposed deep-speare, which represents the state-of-theart for shakespearean sonnet generation. in our experiments, we have adapted and re-trained deep-speare for limerick generation. empirically, it cannot compete with limgen. for handling rhyming constraints, unlike @xcite and @xcite , who generate the last word of each line before generating the rest of the line, our proposed storyline algorithm selects a probability distribution for the last word of each line. beyond poetry generation, templates are often used in other nlp tasks. for biography generation, @xcite noted that a template-guided approach is more interpretable and controllable. @xcite stated that templates are beneficial for guiding text translation. for fostering diversity in generated text, @xcite found that a part-ofspeech template-guided approach is faster and can generate more diverse outputs than the nontemplated diverse beam search of @xcite . limgen's mtbs generates diverse results by design; it also addresses the problem of degradation of performance when beam size grows larger, which has been a challenge noted in several prior works @xcite @xcite . since all rule-based constraints in limgen are easily enforced by a filtering function, it does not need to borrow any advanced techniques from the area of constrained text generation (e.g., @xcite @xcite where constraints are more complicated. ## methodology we first introduce terminology in section 3.1. we present limgen along with amtc in section 3.2. we present the mtbs algorithm in section 3.3, and present our storyline algorithm in section 3.4. ## experiment 4.1 experimental setup to implement limgen, a significant amount of effort has gone into adapting existing nlp technologies for limerick generation. in order to extract pos templates from human written limericks, we modified the pos categories in nltk @xcite by refining certain categories for better quality in our generated limericks. leveraging nltk's pos tagging technique, we obtained a list of refined pos templates from a small limerick dataset of 200 human-written limericks from @xcite and @xcite . for a full list of our modified pos categories see @xcite . since the filtering function f requires knowing each word's syllable and stress pattern, we use cmu (2019) for information on syllable and stress. as for the implementation of the storyline algorithm, there are several existing technologies to indicate whether two words rhyme with each other. for example, deep-speare @xcite proposed a character-level lstm to learn rhyming. for the sake of simplicity and accuracy, we used a rhyming dictionary curated from @xcite . we also used a dictionary of names @xcite from which the storyline algorithm can choose y 1 , the name in the poem's first line. to calculate the semantic similarity between two words, we use the pre-trained word embedding space from spacy's model @xcite . note that algorithm 3.1 is only responsible for generating the last four lines of a limerick. since first lines of limerick usually have a canonical form, we generate the first lines separately. the outline of this section is as follows. we first show why gpt-2-or even retrained gpt-2-cannot produce limericks without limgen. we then show the low-quality output from prior attempts at limerick generation. we have also designed five experiments to compare the quality of limgen's output with limericks from human poets, baseline algorithms, and other state-ofthe-art poetry systems re-purposed for limerick generation. all five experiments were evaluated on amazon mechanical turk by crowd-workers, following a protocol similar to that of @xcite and @xcite (see section 4.4 for details). additionally, an ''expert judgment'' experiment was conducted where more experienced judges directly evaluated the performance of limgen's output and humanwritten limericks across a variety of metrics (see section 4.8 for details). since limgen has three major components: amtc, mtbs, and storyline, we designed three baseline algorithms for an ablation analysis in order to investigate the effectiveness of each of them. in our experiments, limgen and all baseline algorithms use a total beam size of @xmath0, mtbs algorithm's individual beam size per template is @xmath1, and we take the highest scoring poem from the set of output poems. for implementation details please refer to our online github repository @xcite . ## conclusion limgen is the first fully automated limerick generation system. using human judgements, we have shown that our adaptive multi-templated constraints provide limgen with a combination of quality and flexibility. we have shown the value of our diversity-fostering multi-templated beam search, as well as the benefits of our storyline algorithm.