Zero-shot Sonnet Generation with Discourse-level Planning and Aesthetics Features (2022)

Abstract: Poetry generation, and creative language generation in general, usually suffers from the lack of large training data. In this paper, we present a novel framework to generate sonnets that does not require training on poems. We design a hierarchical framework which plans the poem sketch before decoding. Specifically, a content planning module is trained on non-poetic texts to obtain discourse-level coherence; then a rhyme module generates rhyme words and a polishing module introduces imagery and similes for aesthetics purposes. Finally, we design a constrained decoding algorithm to impose the meter-and-rhyme constraint of the generated sonnets. Automatic and human evaluation show that our multi-stage approach without training on poem corpora generates more coherent, poetic, and creative sonnets than several strong baselines.

## introduction a sonnet is a fourteen-line poem with rigorous meter-and-rhyme constraints. in this paper, we aim at generating full-length sonnets that are logically and aesthetically coherent, without training on poetic texts. there are several challenges for this ambitious goal. first, there are limited number of sonnets available to train a fully supervised model. the only resource is a mere 3,355 sonnets collected by @xcite in project gutenberg @xcite , one of the largest free online libraries for english literature. while it is possible to train on related corpus such as general poems or english lyrics @xcite , such approaches are not applicable to many languages for which sizable poetry/lyrics data do not exist. moreover, even if large-scale creative texts exist, learning from and mimicking existing corpora is not creative by definition and is unlikely to result in novel content. figure 1: an overview of our approach. the content planning module generates keywords while maintaining discourse-level coherence. the second module form rhyming pairs and the polishing module enrich the imagination and add poetic flavor. (the keywords underlined in the first step have been polished.) finally, we generate the sonnet with a meter-constrained decoding algorithm. note that all four steps do not require poem/sonnet data. second, coherence remains a known issue among previous works on poetry generation. existing works mainly focus on conforming to the format constraints (i.e., meter-and-rhyme), or generating a small stanza with a typical length of four @xcite @xcite . for full-length sonnets, @xcite propose to use topical words as rhyme words to achieve topical relatedness, but the generated sonnets are not discourse-level coherent. they later generate discourse-level coherent english sonnets through french-english translation @xcite . generating logically and aesthetically ordered poems without relying on content translation from other languages remains a challenge. with all these in mind, we propose zest, a zeroshot sonnet generation model that does not require training on any poetic data. our framework, as is shown in figure 1 , consists of four components: content planning, rhyme pairing, polishing for aesthetics, and final decoding. the first three steps provide salient points for the sketch of a sonnet. the last step is responsible for "translating" the sketch into well-formed sonnets. to achieve zero-shot generation, the content planning and the final decoding components are both trained on a combination of news and story corpora. the trained planning module is aimed to generate several keywords for each sentence to equip the system with general world knowledge to construct a coherent text world. however, the language used by poems is different from that of standard texts because it follows certain rhetorical rhythm and is full of vivid descriptions that appeals to readers' senses and imagination @xcite . to this end, in the polishing step we leverage external knowledge and incorporate two figurative speeches (i.e., simile and imagery) into the planned keywords to boost vividness and imagination. the rhyme and final decoding steps are designed to impose the meter-and-rhyme constraints. while there are previous works on creative generation using the plan-and-write paradigm @xcite @xcite @xcite , they all rely on training data from the target task domain (e.g., use story data to train storyline-planning). we on the other hand adopt content planning to disentangle the training from the decoding step to circumvent the shortage of training data for poetry generation. we summarize our contributions as follow: ## background in this section, we introduce the characteristics of sonnets in terms of structure, meter and rhyme. we then define important terminologies. ## approach overview as is shown in figure 1 , our sonnet generation model can be divided into four steps. at step a, we train a title-to-outline module by finetuning t5 @xcite on keywords extracted from news reports and stories. during inference time, we generate a fourteen-line sonnet sketch that contain those content words c (section 3.1). at step b, we aim at forming the correct rhyming pairs. we first select the initial rhyme words from c i for @xmath0, and then generate the remaining rhyme words (i.e., for @xmath1 generate keywords given the title: prince andrew loses military titles. prince, stop, highness. face, assault, allegation. … huge, rapid, retreat. a. title: prince andrew loses military titles. line 1: [k 11 , k 12 , k 13 ]. line 2: [k 21 , k 22 , k 23 ]. … line n: [k n1 , k n2 , k n3 ]. title: prince andrew loses military titles. line 1: [prince, stop, highness]. line 2: [face, assault, allegation]. … line n: [huge, rapid, retreat]. b. and slant rhyme words (section 3.2). at step c, we infuse imagery and simile as two figurative devices to c (section 3.3). in the last step, we leverage a fine-tuned language model with constrained decoding algorithm to impose the meter-and-rhyme constraints (section 3.4). ## conclusion we investigate the possibility of generating sonnets without training on poems at all. we propose a hierarchical planning-based framework to generate sonnets which first plans the high-level content of the poem, refine the predicted keywords by adding poetic features, and then achieve decoding-time control to impose the meter-and-rhyme constraints. extensive automatic and expert evaluation show that our model can generate sonnets that use rich imagery and are globally coherent, poetic, and emotion provoking.