SlowGuess/NAACL_2022_OCR huggingface dataset:
Yufei Tian
Computer Science Department, University of California, Los Angeles yufeit@cs.ucla.edu
Nanyun Peng
Computer Science Department, University of California, Los Angeles violetpeng@cs.ucla.edu
Poetry generation, and creative language generation in general, usually suffers from the lack of large training data. In this paper, we present a novel framework to generate sonnets that does not require training on poems. We design a hierarchical framework which plans the poem sketch before decoding. Specifically, a content planning module is trained on non-poetic texts to obtain discourse-level coherence; then a rhyme module generates rhyme words and a polishing module introduces imagery and similes for aesthetics purposes. Finally, we design a constrained decoding algorithm to impose the meter-and-rhyme constraint of the generated sonnets. Automatic and human evaluation show that our multi-stage approach without training on poem corpora generates more coherent, poetic, and creative sonnets than several strong baselines. $^{1}$
A sonnet is a fourteen-line poem with rigorous meter-and-rhyme constraints. In this paper, we aim at generating full-length sonnets that are logically and aesthetically coherent, without training on poetic texts.
There are several challenges for this ambitious goal. First, there are limited number of sonnets available to train a fully supervised model. The only resource is a mere 3,355 sonnets collected by Lau et al. (2018) in Project Gutenberg (Hart, 2004), one of the largest free online libraries for English literature. While it is possible to train on related corpus such as general poems or English lyrics (Ghazvininejad et al., 2016), such approaches are not applicable to many languages for which sizable poetry/lyrics data do not exist. Moreover, even if large-scale creative texts exist, learning from and mimicking existing corpora is not creative by definition and is unlikely to result in novel content.
a. Content Planning (trained on news and stories)
Line 1: recall, time, consider
Line 2: dark, gaze, stars
Line 3: day, bright, [R3]
Line 4: fog, white, [R4]
Line 13: know, youth, trust
Line 14: romantic, love, [R14]
b. Rhyme Pairs Generation (the CMU dictionary)
Rhyme Scheme: ABAB CDCD EFEF GG
Line 1: Recall, time, consider
Line 3: day, bright, glitter
Line 4: fog, white, Mars
c. Polishing for Aesthetics (Imagery pairs, simile phrases)
Imagery:
Line 3: day  $\rightarrow$  sun
Line 14: love  $\rightarrow$  rose
Simile:
Line 3: bright like diamond  $(\mathrm{/x / x})$
Line 4: white like snow (/x/)
d. Sonnet Generation (trained on news and stories)
Recall the time and having me consider,
Figure 1: An overview of our approach. The content planning module generates keywords while maintaining discourse-level coherence. The second module form rhyming pairs and the polishing module enrich the imagination and add poetic flavor. (The keywords underlined in the first step have been polished.) Finally, we generate the sonnet with a meter-constrained decoding algorithm. Note that all four steps do not require poem/sonnet data.
Discourse-level
Meter-Constrained Decoding
Second, coherence remains a known issue among previous works on poetry generation. Existing works mainly focus on conforming to the format constraints (i.e., meter-and-rhyme), or generating a small stanza with a typical length of four (Lau et al., 2018; Liu et al., 2019; Yi et al., 2020). For full-length sonnets, Ghazvininejad et al. (2016) propose to use topical words as rhyme words to
1Our code and data are available at https://github. com/PlusLabNLP/Sonnet-Gen.
Human Language Technologies, pages 3587 - 3597
achieve topical relatedness, but the generated sonnets are not discourse-level coherent. They later generate discourse-level coherent English sonnets through French-English translation (Ghazvininejad et al., 2018). Generating logically and aesthetically ordered poems without relying on content translation from other languages remains a challenge.
With all these in mind, we propose Zest, a Zero-shot sonnet generation model that does not require training on any poetic data. Our framework, as is shown in Figure 1, consists of four components: content planning, rhyme pairing, polishing for aesthetics, and final decoding. The first three steps provide salient points for the sketch of a sonnet. The last step is responsible for "translating" the sketch into well-formed sonnets.
To achieve zero-shot generation, the content planning and the final decoding components are both trained on a combination of news and story corpora. The trained planning module is aimed to generate several keywords for each sentence to equip the system with general world knowledge to construct a coherent text world. However, the language used by poems is different from that of standard texts because it follows certain rhetorical rhythm and is full of vivid descriptions that appeals to readers' senses and imagination (Gibbs Jr et al., 1994). To this end, in the polishing step we leverage external knowledge and incorporate two figurative speeches (i.e., simile and imagery) into the planned keywords to boost vividness and imagination. The rhyme and final decoding steps are designed to impose the meter-and-rhyme constraints.
While there are previous works on creative generation using the plan-and-write paradigm (Wang et al., 2016; Martin et al., 2018; Peng et al., 2018; Yao et al., 2019; Gao et al., 2019; Goldfarb-Tarrant et al., 2019), they all rely on training data from the target task domain (e.g., use story data to train storyline-planning). We on the other hand adopt content planning to disentangle the training from the decoding step to circumvent the shortage of training data for poetry generation. We summarize our contributions as follow:
- We propose Zest, a Zero-Shot sonnet generation framework, by disentangling training from decoding. Specifically, we first learn to predict context and rhyme words from news and story dataset, and then polish the predicted keywords to promote creativity. A constrained decoding algorithm is designed to impose the meter-and-rhyme
constraints while incorporating the keywords.
- We develop two novel evaluation metrics to measure the quality of the generated poems: automatic format checking and novelty evaluation (i.e., diversity and imageability).
- Human evaluation shows that Zest generates more discourse-level coherent, poetic, creative, and emotion-evoking sonnets than baselines.
In this section, we introduce the characteristics of sonnets in terms of structure, meter and rhyme. We then define important terminologies.
2.1 The Structures of Sonnets
We aim to generate the two most representative sonnets: Shakespearean and Petrarchan. Sonnets make use of rhymes in a repeating pattern called rhyme schemes as shown in Table 1. For example, when writing a Shakespearean sonnet, poets usually adopt the rhyme scheme of ABABCDCEFEGG. Although all sonnets have 14 lines, a Petrarchan sonnet consists of an 8-line stanza called an octave followed by a 6-line stanza called a sestet. On the other hand, a Shakespearean sonnet consists of three 4-line quatrains and a 2-line rhyming couplet which leaves the reader with a lasting impression.
# of Lines	Iambic Penta	Structure	Rhyme Scheme
Shakespearean Sonnet	14	Yes	3 quatrain 1 couplet	ABAB CDCD EFEFGG
Petrarchan Sonnet	14	Yes	1 octave 1 sestet	ABBA ABBA CDECDE
Table 1: Comparison between a Shakespearean sonnet and a Petrarchan sonnet.
2.2 Meter Constraints
Most sonnet conform to iambic pentameter, a sequence of ten syllables alternating between unstressed (x or da) and stressed syllables (/ or DUM). Strictly speaking, each line reads with the rhythm  $(\mathrm{da} - \mathrm{DUM})^5$ , which enhances the tone for the poem and operates like an echo. In reality, there are many rhythmic variations. For example, the first foot is often reversed to sound more assertive, and can be written as  $(\mathrm{DUM} - \mathrm{da} * (\mathrm{da} - \mathrm{DUM})^4)$ . Another departure from the standard ten-syllable pattern is to append an addition unstressed syllable to the end, forming feminine rhymes which can be written as  $((\mathrm{da} - \mathrm{DUM})^5 * \mathrm{da})$ .
2.3 Rhyme Words, Couplets and Patterns
A pair of rhyme words consists of two words that have the same or similar ending sound. A rhyming couplet is a pair of rhymed lines. For example, Line 1&3, 2&4 in Figure 1 are two pairs of rhyming couplets. From the CMU pronunciation dictionary (Weide, 1998), we know that "fall" and "thaw" in Figure 1 are strict rhyming pairs because they have exactly the same phonetic endings: "AO L". "Leaves" ("IY V Z") and "trees" ("IY Z") are slant rhymes, because they have the same stressed vowels, while the ending consonants are similar but not identical.
2.4 Terminology
We formally define the following terms:
Keywords  $\kappa$  : content words and rhyme words combined. They contain main ideas of a poem and define the rhyming pattern.
- Content words  $\mathcal{C}$ : keywords that do not appear in the end of each line. We target at predicting 2 context words per line,  $C_{i1}$  and  $C_{i2}$ .
- Rhyme words  $\mathcal{R}$ : words in the end of each line. For example, in a Shakespearean sonnet with the rhyme scheme ABABCDCEFEGG, there are seven pairs of rhyme words:  $R_{1}R_{3}$ ,  $R_{2}R_{4}$ , ..., and  $R_{13}R_{14}$ .
- Initial rhyming lines  $\mathcal{I}_{Init}$ : index of the lines that the first rhyme word in a rhyming couplet appears (e.g.,  $\mathcal{I}_{Init} = [1, 2, 5, 6, 9, 10, 13]$  for a Shakespearean sonnet and  $\mathcal{I}_{Init} = [1, 2, 9, 10, 11]$  for a Petrarchan sonnet).
- Sketch: The sketch of a poem contains three aspects: 1) content words that cover the key concepts or main ideas, 2) the rhyme words to appear at the end of each line, and 3) the modification of keywords for aesthetics.
Overview As is shown in Figure 1, our sonnet generation model can be divided into four steps. At step a, we train a title-to-outline module by finetuning T5 (Raffel et al., 2019) on keywords extracted from news reports and stories. During inference time, we generate a fourteen-line sonnet sketch that contain those content words  $\mathcal{C}$  (Section 3.1). At step b, we aim at forming the correct rhyming pairs. We first select the initial rhyme words from  $\mathcal{C}_i$  for  $i\in \mathcal{I}_{Init}$ , and then generate the remaining rhyme words (i.e., for  $i\in \overline{\mathcal{I}_{Init}}$ ) by forcing the decoder to sample from a vocabulary pool that contains strict
Figure 2: A comparison diagram of two input-output formats to train the first module. While format A is most straightforward, there is no control over the output structure. Therefore, we purposefully design the prompt shown in format B to control the number of keywords and the number of lines to be generated.  $\kappa_{ij}$  represents the mask tokens at the  $i$ -th sentence.
and slant rhyme words (Section 3.2). At step c, we infuse imagery and simile as two figurative devices to  $\mathcal{C}$  (Section 3.3). In the last step, we leverage a fine-tuned language model with constrained decoding algorithm to impose the meter-and-rhyme constraints (Section 3.4).
3.1 Content Planning
For each piece of news or stories, we train a title-to-keywords framework that predicts the outline. To this end, we first extract three most salient words per line using the RAKE (Rose et al., 2010) algorithm, which is a domain-independent keyword extraction technique.
Controllable Text Formatting We then leverage the task adaptability of the pretrained T5 (Raffel et al., 2019) to predict the keywords of the whole body. As a unified framework that treats every text processing task as a "text-to-text" problem, T5 can be easily adapted to our task as shown in Figure 2.A, where the input is an instruction to generate the sketch given the title, and the outputs are multiple keywords for each line. However, we need a mechanism to specify the number of lines and keywords to be generated, since we train on prosaic texts with varying formats but infer only on the 14-line sonnets.
To solve this problem and gain control over the poem structures, we format the input and output as shown in Figure 2.B. Specifically, we use [MASK] tokens as placeholders for the keywords. Now that one [MASK] token on the input side corresponds to exactly one word on the output side, we are able to specify the number of lines and keywords during the inference time.
"Title: The Four Seasons.
Keywords: [plums, autumn, leaves].
Keywords: [trees, quivering, fall].
Keywords: [tangled, branches,  $\mathbb{R}_3$ ].
Keywords: [fog, snow,  $\mathbb{R}_4$ ].
Keywords: [blossoming, lemon, fell].
Keywords: [swirl, air,  $R_{14}$ ].
Figure 3: An example input to query the remaining rhyme words during the inference time. Rhyme words in the same background color form a rhyming pair.
3.2 Generating Rhyme Words
Our title-to-outline model is trained to generate keywords, regardless of the rhyme constraints. In this section, we describe the procedure to generate rhyme pairs. Specifically, we force the model to generate a 14-line outline, with two or three content words for each line depending on whether the line is an initial rhyming line:
$$
\operatorname {K e y w o r d s} _ {i} = \left\{ \begin{array}{l} \left[ K _ {i 1}, K _ {i 2}, K _ {i 3} \right], \text {i f} i \text {i n} \mathcal {I} _ {\text {I n i t}} \\ \left[ K _ {i 1}, K _ {i 2} \right], \text {o t h e r w i s e .} \end{array} \right. \tag {1}
$$
where  $K_{ij}$  represents the  $j$ -th keyword in the  $i$ -th line. Among the three keywords in the initial rhyming lines, we select the last word as the initial rhyme word.
Rhyme Pairs Generation Given the initial rhyme words, we then retrieve all the possible rhyme words  $\mathbb{R}$  based on their phonetics information from the CMU pronunciation dictionary (Weide, 1998). This include strict rhymes and slant rhymes. For instance, in Figure 3, the retrieved rhyme word candidates  $\mathbb{R}$  for 'leaves' are ['achieves', 'believes', 'Steves', 'trees', ...]. The probability distribution for generating the rhyme word  $w_{R}$  from the candidate list  $\mathbb{R}$  is modified as:
$$
P ^ {\prime} \left(w _ {R}\right) = \left\{ \begin{array}{c c} \frac {p \left(w _ {R} \mid \text {c o n t e x t}\right)}{\sum_ {x \in \mathbb {R}} p \left(x \mid \text {c o n t e x t}\right)} & , \text {i f} w _ {R} \in \mathbb {R} \\ 0 & , \text {o t h e r w i s e .} \end{array} \right. \tag {2}
$$
where  $p(w_{R}|\cdot)$  is the original word probability yielded by the title-to-outline decoder.
3.3 Polishing Context Words for Aesthetics
Now, we have the generated context words and rhyme words that are discourse-level coherent yet less vivid. To this end, we use external knowledge to incorporate two figurative devices into the planned keywords: imagery and simile.
Imagery We leverage the <symbol, imagery> pairs (e.g., <love, rose>) in the ConceptNet knowledge base (Liu and Singh, 2004) and finetune a
imagery generation model from a pretrained model called COMmonsEnse Transformer (Bosselut et al., 2019) (COMMeT). It is trained on imagery pairs to generate the imagery word given the symbolism word as input. At inference time, we randomly sample multiple nouns from the sketch to predict their imageries, and only make replacement for the two most confident generations. For example in Figure 1, both <day, sun> and <love, rose> are generated, yet we only replace 'love' with 'rose', because the probability of generating the latter pair is much higher than the former pair.
Simile A simile phrase consists of two parts: the adjective and the figurative vehicle. For example, 'sudden like a flash' is a simile phrase where 'a flash' is the figurative vehicle of 'sudden'. We leverage the simile generation model by Chakrabarty et al. (2020) as an off-the-shelf tool<sup>2</sup> to generate simile vehicles from adjectives to extend the sketch keywords. At inference time, we randomly sample multiple adjectives from the sketch to predict their figurative vehicles, and only keep the most confident ones. In addition, we also make sure the generated simile phrase conforms to the iambic-meter constraint. For example in Figure 1, the phrase 'bright like diamond'  $(/\mathrm{x} / \mathrm{x})$  follows the iambic meter, whereas another phrase such as 'shining like diamond'  $(/\mathrm{xx} / \mathrm{x})$  will be disregarded.
3.4 Sketch to Sonnet Generation
In order to write fluent and poetic languages that meet the meter-and-rhyme constraints, we make the following adaptations. First, generating the full sonnet requires more powerful pretrained model than generating the outlines. Therefore, we fine-tune GPT-Neo-2.7B on the same combination of news and stories data as a language model to generate the sonnet. Second, to effectively incorporate the rhyme words at the end of each line, we follow previous methods (Ghazvininejad et al., 2016; Van de Cruys, 2020) and generate the whole sonnet line-by-line in reverse, starting from the final rhyme word to the first word. That is to say, our language model is finetuned to generate from right to left to better enforce rhyming. Third, we include the sketch in the prompt, so that the decoder will learn to give higher probability for these keywords. We then use lexically constrained decoding similar to that of Grid Beam Search (Hokamp and Liu, 2017) to incorporate the keywords. In addition, we also include the previously generated lines in the prompt to generate the next line in a sonnet to promote discourse-level coherence. A simile phrase in the sketch is considered fixed that cannot be modified. Namely, we force to generate the whole phrase when the first word in the phrase is decoded. Lastly, we modifies the beam search algorithm to impose the meter-and-rhyme constraint. Algorithm 1 displays the skeleton of our decoding strategy. At each decoding step, we apply rhythm control, so that only those tokens that satisfy the iambic-pentameter and its two variations (listed in Section 2.2) are kept in the beams. We recursively generate the next token until 10 or 11 syllables are generated and make up a metric line where all the context words are incorporated.
Our approach does not require poem data. The training dataset for the content planing module and the decoding module is a combination of 4,500 CNN news summary (Hermann et al., 2015) and 16,000 short stories crawled from Reddit.<sup>3</sup> We remove those articles that contain conversations,urls, or are too long (>50 lines) or too short (<8 lines). During decoding, we generate sonnets using
top-k sampling and set norepeat_ngram_size to 3 to promote creativity and avoid repetition.
We finetune the pretrained T5 for 10 epochs for the "content planning" component, and finetune GPT-Neo-2.7B for 6 epochs for the decoding component. We use one Nvidia A100 40GB GPU. The average training time is  $5\sim 10$  hours for each experiment.
Hafez A program that is trained on lyrics data and generates sonnets on a user-supplied topic (Ghazvininejad et al., 2018). It combines RNNs with a finite state automata to meet the meter and rhyme constraints. Hafez is the state-of-the-art model that generates full-length sonnets but it does not train on standard, non-poetic texts.
Few-shot GPT-3 We utilize the most capable model in the GPT-3 family (Brown et al., 2020),  $GPT3$ -davinci<sup>4</sup>, as a strong baseline to follow instructions and generate sonnets. In the prompt, we provide two examples of standard sonnets and then instruct the model to generate a sonnet given the title. We force the output to be exactly 14 lines.
Ablations of our own model To test the effectiveness of our sketch-before-writing mechanism, we also compare variations of our own model:
Prosaic An stronger version of nmf (Van de Cruys, 2020), the first (and only) model to generate rhyming verses from prosaic texts. Topical and rhyme consistency are achieved by modifying the word probability of rhyme and topical words. For fair comparison, we replace the original vanilla encoder-decoder with GPT2 that Zest is finetuned on, and force the output to be 14 lines. Model comparison between Prosaic and Zest serves as ablations of the keyword-planning component (versus end-to-end generation).
Zest  $w / o$  fig The model consisting of step a, c, and d as illustrated in Figure 1, but without the polishing the sketch for figurative devices. Our full model consisting of 4 modules is called Zest.
4.3 Decoding Strategy
For decoding, we generate sonnets from our models using a top-k random sampling scheme where  $k$  is set to 50. At each time step, the GPT2 model generates subwords instead of complete words. In order
$^{2}$ https://github.com/tuhinjbcse/SimileGeneration-EMNLP2020
<sup>3</sup>https://www.reddit.com/r/shortscarystories/
<sup>4</sup>https://beta.openai.com/docs/engine
to impose the meter and rhyme constraints while decoding for each word, we ask the language model to continue to generate until a complete word is generated as indicated by special space token 'G'. To avoid repetition and encourage creativity, we set norepeat_ngram_size to 3 and use a softmax temperature of 0.85.
It is difficult and thus uncommon to automatically evaluate the quality of poems. For example, Ghazvininejad et al. (2016) and Van de Cruys (2020) exclude automatic evaluation, with the later stating "Automatic evaluation measures that compute the overlap of system output with gold reference texts such as BLEU or ROUGE are of little use when it comes to creative language generation." In addition, Yang et al. (2021) show current metrics have very low correlation with human. Hence, we propose to evaluate the generated poems in two novel aspects: format and novelty.
Format Checking For rhyme checking, we count the percentage of rhyme pairs that belong to strict or slant rhymes. For meter checking, we consider the following most common scenarios mentioned in Section 2.2: the standard Iambic Pentameter; the first foot reversed; and a feminine rhyme. In all scenarios, words that are monosyllables can serve as both stressed and unstressed syllables. For a looser standard, we also calculate the percentage of valid lines that contain either 10 or 11 syllables.
Novelty We follow the settings in existing works Yi et al. (2018, 2020) and calculate the Distinct-2 scores (Li et al., 2015) to measure the diversity of generated poems. Besides, imagery is another important feature of poems as pointed out by linguistic studies Kao and Jurafsky (2012); Silk (2006). Here, we calculate Imageability score to assess how well a poem invokes mental pictures of concrete objects. Specifically, we extracted the features from the resource by Tsvetkov et al. (2014), who use a supervised learning algorithm to calculate the imageability ratings of 150,114 terms. For each poem, we average the ratings of all its words after removing the stop words.
4.5 Human Expert Judgement
Considering the expertise required to appreciate sonnets, we recruit 6 professionals that hold a bachelor's degree in English literature or related majors as domain experts to annotate the generated sonnets.
Model Name	Format Checking	Novelty
Rhyme	Meter	Syllable	Dist-2	Img
Hafez	98.3%	76.8%	95.7%	84.8	0.44
Fewshot GPT-3	14.0%	17.6%	30.9%	85.3	0.48
Prosaic	100%	10.1%	19.0%	84.9	0.46
Zest w/o fig	100%	77.7%	98.6%	86.6	0.49
Zest	100%	75.6%	98.4%	86.6	0.51
Human	94.6%	70.7%	81.8%	87.4	0.52
Table 2: Automatic evaluation results for rhyme, meter, syllable checking, distinct scores, and imageability (Img in the table). Best machine scores are underlined.