GP oe T : a Language Model Trained for Rhyme Generation on Synthetic Data (2023)

Abstract: Poem generation with language models requires the modeling of rhyming patterns. We propose a novel solution for learning to rhyme, based on synthetic data generated with a rule-based rhyming algorithm. The algorithm and an evaluation metric use a phonetic dictionary and the definitions of perfect and assonant rhymes. We fine-tune a GPT-2 English model with 124M parameters on 142 MB of natural poems and find that this model generates consecutive rhymes infrequently (11%). We then fine-tune the model on 6 MB of synthetic quatrains with consecutive rhymes (AABB) and obtain nearly 60% of rhyming lines in samples generated by the model. Alternating rhymes (ABAB) are more difficult to model because of longer-range dependencies, but they are still learnable from synthetic data, reaching 45% of rhyming lines in generated samples.

## introduction the quality of texts generated by language models (lm) has improved tremendously in recent years. while their factual accuracy is still open to debate, this is not an issue when using lms with a creative purpose, in particular to generate works of art such as poems. in the recent past, lms were put to use for poetry generation in several studies @xcite @xcite @xcite , which found that fluency and intelligibility reached satisfactory levels. however, poems often exhibit structural, text-level properties that are still quite difficult to manage by lms: rhyming patterns and division into verses and stanzas. while not all poems make use of these properties, a convincing lm for poetry generation should be able to deal with them. in this paper, we focus on the first property and propose a method to adapt an lm so that it generates rhyming verses, with modest computing requirements. we start from an unconstrained au-toregressive lm, in our case gpt-2, which we fine-tune first on a poetry corpus of about 120 mb to improve its style (section 3). we design a rulebased system which modifies text generated by the lm so that it obeys a given rhyming pattern while retaining acceptable fluency, and we generate two datasets of 160k lines (6 mb) each with the aabb and abab patterns (section 4). we further finetune the lm on these synthetic datasets in order to generate rhyming verses with the respective patterns, thus showing that they can be learned by a moderately-sized lm (section 5). we also introduce a rhyming metric (see section 2) based on an english rhyming dictionary, and use it throughout the study to count the proportion of perfect and assonant rhymes generated by a model. we find that this is very low (11%) for the lm fine-tuned on natural poetry with variable rhyming patterns, but increases to around 60% when the lm learns only the aabb pattern from synthetic data. the abab pattern is more challenging, but can still be learned, reaching around 45% rhyming lines. in the conclusion (section 7), we discuss some issues related to the integration of the rhyming lms into an existing, operational system for interactive poetry generation. 1 our contributions are the following: ## measuring the number of rhymes a criterion for measuring the number of rhyming verses is key for the present study. we present a metric that distinguishes between perfect rhymes, assonant rhymes, and no rhymes, using a rhyming dictionary derived from an english pronunciation dictionary. we test it on a corpus of human poetry annotated for rhyme and show that its accuracy is sufficient for use in this study. ## an auto-regressive language model fine-tuned on poetry our starting point is gpt-2 @xcite , a general-purpose decoder lm for english. we use the python implementation provided by the huggingface library @xcite . 7 we enable the model to generate poetry by fine-tuning it first on a corpus of english poetry (3.1), and then by designing constraints so that its output has the form of a poem, with lines and stanzas (3.2). we evaluate the frequency of rhymes in the output of this model using our metric (3.3), before moving on to its specific training for rhyming in the next sections. ## synthetic data with rhymes: rule-based generation we use a rule-based approach to modify the poems generated by the previous model so that they follow a given rhyme scheme, which is specified in conventional form (e.g. aabb, abab or abba). this is part of our earlier interactive system for poetry generation (popescu-belis et al., 2022) which combines lms with rules governing form, rhymes, topics and emotions. the rule-based rhyming algorithm parses the scheme, and for every second line of a rhyme (e.g., given aabb, for the second and fourth lines), it modifies the last word so that it rhymes with the last word of the previous line. the inverted rhyming dictionaries presented in section 2.2 and the finetuned gpt-2 model are used as follows. the algorithm obtains from the first dictionary the perfect rhyme ending the word to replace, and it searches the second dictionary for all the words that share this perfect rhyme. if none is found, the words sharing the respective assonant rhyme are used instead. each word is inserted in the entire line and the result is submitted to gpt-2, which generates a likelihood score for each of these sequences. the replacement word leading to the highest score is selected. therefore, to generate rhyming poems, we first generate a non-rhyming one and then we re-generate the last words so that they rhyme according to the given patters. using this strategy, we generate large numbers of poems, first with the aabb rhyming pattern, and later with the more challenging abab pattern. for each pattern we generate 20,000 quatrains (fourline stanzas) resulting in about 6 mb of text. some cleaning of the data is necessary because some lines are made mostly of punctuation or include special characters. about 0.04% of the lines are removed. to simplify training, we insert a blank line after lines aa and then bb of the quatrain, so that the training data is made of rhyming couplets only. alternatively, to learn abab, we insert a blank line after each quatrain. our metric found that the first dataset has a rhyming accuracy of 97.8%, which is expected because the rhyming algorithm and the metric make use of the same dictionary. moreover, as the lm must capture dependencies between words at the end of lines regardless of the punctuation, we hypothesize that if we remove punctuation at the end of the verses in the training dataset, the lm would better learn rhyming patterns. the results below confirm this hypothesis. ## related work before the advent of deep neural lms, various combinations of rule-based approaches and n-gram lms have been tried. for instance, @xcite defined a poem generation system which included a phonological model "to impose a sense of prosody" but not dealing with rhymes. in fact, rhyming was not considered the most urgent problem to solve as lms were struggling with fluency and, especially, meaning. large neural lms have brought high expectations regarding their capacities to generate structured texts such as poems, and clearly improved fluency for high-resource languages. poem generation with gpt-2 @xcite was discussed, for instance, by @xcite in a blog entry shortly after the model was made available. more recently, chatgpt (openai, 2022) has tremendously improved the quality and rele-vance of generated text. however, anecdotal evidence shows that it cannot reliably generate a given rhyming pattern. 11 @xcite , an even larger lm, is likely to improve this capability, as initial analyses seem to show @xcite . lms based on recurrent neural networks (rnns) were trained by @xcite on 1.5 mb of english sonnets, first with a single phonetic model and hmm-based phonetic-toorthographic transliteration, and then with decoupled models for content vs. form. rhymes from the first model were exemplified, but not evaluated, while the second approach targeted only rhythm, but not rhyme. for chinese, one of the earliest systems using 11 when asked 'what are the possible rhyming patterns?", chatgpt enumerates several patterns with definitions and examples, but with factual mistakes such as "abab: in this pattern, each line rhymes with the line that comes after it." moreover, the example generated by chatgpt for the abab pattern is an abcb stanza. when prompted to "write one quatrain about the ocean, make the first verse rhyme with the third one, and the second with the fourth", chatgpt generates three fluent quatrains, but with incorrect rhyming patterns (aaab, ccde, and ffaa). moreover, chatgpt seems unable to reliably generate verses (or even plain sentences) with a fixed number of syllables or words larger than about 7. rnns was proposed by @xcite , starting from user-provided keywords and generating a quatrain line-by-line, with pre-defined line lengths and tonal patterns. rhyming is only enforced between the second and fourth lines, simply by disallowing the decoder to select ending characters that do not rhyme. the constraints are similar to the method of @xcite who used a generative summarization approach. @xcite built a chinese poem generator using a variational encoder and adversarial training, starting from a title. poems were evaluated for topic consistency, fluency, and meaning, but not explicitly for rhyming. @xcite studied the problem of generating a poem from prose and compared lstm to transformer models, but did not model explicitly rhymes, nor evaluated them. poetryme is a rule-based interactive poem generation system initially designed for portuguese and later extended to spanish and english @xcite . in the interactive version, foot_9 assistance is provided to users for selecting end-of-line words that rhyme, through a dictionary. in the standalone twitter bot (@poetartificial), candidates which happen to contain rhyming lines more than others are rewarded. poem machine @xcite is an assistant for finnish, which provides help for rhyming via a phonetic dictionary, but does not select rhyming words automatically. our own cr-po system for french @xcite , combined a general lm with topic and emotion-specific lms, and with rules for constraining form and rhymes (the latter are used in this paper). hafez was one of the first systems to combine interaction and deep neural lms @xcite . the system gets the desired features from the user, including keywords and sentiment, transforms them into transducers, and uses a rnn filtered by these transducers to generate a quatrain. rhyming words are generated early in the process, using word2vec similarity and a phonetic representation, typically in an abab pattern, and afterwards they constrain the generation of the poem. henceforth, rhyming is always ensured. @xcite proposed a rnn encoder-decoder architecture with attention, with grus, for english and french poems. the model is trained to generate a line of poetry given the preceding one, with a decoder part that models the new line in reverse order. the advantage of starting from the last word, as for hafez, is that it can be sampled with a probability distribution that incorporates rhyming constraints, using a rhyming dictionary similar to ours, with an additional bias to avoid repeating the consonant group preceding the final vowel [+ consonant]. in the experiments, the abab cdcd pattern is always used. human judges ranked a set of 40 generated poems almost as high as human ones on several parameters. no scores are provided for rhyming alone, likely because it is nearly perfect given the architecture, but the rhyming component improved scores of 'poeticness' and human-likeliness. poelm @xcite uses a decoder (gpt-style with 350m parameters) to learn rhythm and syllables from a large corpus of prose in spanish and basque. input text is segmented into phrases, and for each set of phrases of a sentence a set of tags is prepended to the sentence, e.g. <len:11><end:@xmath0. poelm learns these control tags and can leverage them to generate lines of poetry of desired length and endings. however, as the model does not learn rhyming rules (i.e. identity of syllables) but only identifies actual syllables, poem generation must start by specifying exactly the ending syllables of each line. evaluation is done by completing the initial line of human poems with poelm, and then asking human judges which version they prefer. bygpt5 @xcite is a character-level transformer-based decoder, with generation conditioned on rhyme, meter, and alliteration. the model is initialized on the decoder of byt5 @xcite , trained on large amounts of data, and then fine-tuned on a machine-labeled corpus of pseudo-quatrains in english and german, separately. meter and rhyme are evaluated with classifiers trained on labeled data. overall, according to automatic and human measures, bygpt5 produces better results than byt5 and subwordlevel models such as gpt-2 and mt5. ## discussion and conclusion the rhyme-generating lm presented here, gpoet, is intended for integration in our interactive poem generation system @xcite . while the experiments above show that rhyming patterns can be learned from synthetic data, several issues remain to be solved in future studies. our rule-based rhyming algorithm operates on a poem already generated by a lm with several other parameters as input, e.g. a title or first verse, a desired theme or emotion, and a poetical form (such as a sonnet). we must now integrate gpoet in this pipeline, and ensure that the generated rhymes are not altered by the other constrains of the system. moreover, we must ensure that the lexical diversity of gpoet is not reduced by its training on synthetic data. we intend to address the problem of generating a desired form using the rule-based algorithm presented in section 3.2, which takes advantage of a maximum length for the lm decoder. it may seem straightforward to replace gpt-2 with gpoet in this algorithm, in order to obtain rhyming lines of a desired length, but our initial experiments have shown that rhymes are less satisfactory when the desired length is very different from the synthetic data gpoet was trained on. moreover, while our rule-based rhyme generator can be easily adapted to any rhyming pattern, this is not yet the case for gpoet, which is trained on one pattern at a time in our proof-of-concept. the solution lies probably in using a labeling system to indicate which lines must rhyme, and then training a gpoet model to learn the effects of labels rather than a single rhyming pattern, in the style of the ctrl model @xcite . in this paper, we demonstrated that rhyming is learnable with lms that can be efficiently finetuned and queried with very moderate computing requirements. the key to effective fine-tuning is the use of synthetic data, which we showed how to generate in much larger amounts than what human poets have ever written. however, not all rhyming patterns are learned equally well: a pattern that exhibits longer-term dependencies such as abab is harder to learn than a more local one such as aabb. overall, lms that are able to deal with rhyme, and later with form, are part of our ongoing effort to design an interactive poetry generator, with the aim of enhancing (but not replacing) human creativity.