The Mechanical Bard: An Interpretable Machine Learning Approach to S hakespearean Sonnet Generation (2023)

Abstract: We consider the automated generation of sonnets, a poetic form constrained according to meter, rhyme scheme, and length. Sonnets generally also use rhetorical figures, expressive language, and a consistent theme or narrative. Our constrained decoding approach allows for the generation of sonnets within preset poetic constraints, while using a relatively modest neural backbone. Human evaluation confirms that our approach produces Shakespearean sonnets that resemble human-authored sonnets, and which adhere to the genre’s defined constraints and contain lyrical language and literary devices.

## introduction we consider the task of automatically generating shakespearean sonnets, a popular poetic form with highly specific rhyme and meter constraints 1 . each sonnet consists of three quatrains followed by a single couplet according to the rhyme scheme abab bcbc cdcd ee, and each line contains ten syllables with a stress pattern of iambic pentameter. rather than train a model to obey these constraints implicitly (which leads to enormous models that that still do not obey the constraints), we opt to enforce them explicitly using a simple but novel approach to generation. in particular, we use part-of-speech (pos) templates selected and edited from individual lines in shakespeare's sonnets, with each template intended to offer a different combination of parts of speech and narrative directions. associated thematic words are then selected and placed at the end of each template, and their rhyming pairs are chosen dynamically by a language model (e.g., @xcite and placed at the end of the corresponding lines according to the rhyme scheme. the sun of heaven on a golden day to burn the earth's fire by the flame and spread where all the flowers of your fair days lay. these are the blossoms that you take care of. why do you linger such a long delay? forgive the fluttered flower of meek love or who you have so long to love the day? the joys of love, the beauty on the face, shall be your fate and be your own delight. you have the beauty of your own embrace. you cannot reminiscence. cannot write. between the living and the deadening breath. you go the way of your beloved death. figure 1 : a sonnet generated with the theme "death". the rest of the line is filled with related words that fit the specified pos and meter, leading to the end rhyme word. figure 1 shows sample output. our use of these templates ensures sophisticatedseeming language and syntax that competing systems do not capture. our approach provides excellent grammatical structure comparable to that of human-written poetry, all while using a relatively simple model and generation procedure. we extensively evaluate the ability of our approach to generate whole sonnets (a setting often ignored by recent work in poetry generation) and find that our approach is preferred over strong baselines by both expert annotators (recruited from an academic english department) and by crowdworkers. as this research was conducted before the release of chatgpt, we were not able to robustly compare our model's performance against this language model. however, we make several observations about the poetic quality of sonnets generated by chatgpt. ## related work early attempts at poetry generation relied mainly on rule-based methods @xcite @xcite . more recent automated poetry generation techniques, especially for sonnet generation, have relied on combinations of task-specific language models and rules. for instance, ghazvininejad et al. (2016)'s hafez uses a finite state acceptor to generate a large number of possible lines, the best of which are then selected with an rnn trained on song lyrics. like our approach, they use rhyming dictionaries to find rhyming words and word embeddings to find topical words. similarly, @xcite preselects rhyming words and generates lines backwards with a recurrent neural network (rnn). also in this vein are lau et al. (2018)'s deepspeare, which consists of an lstm language model, an iambic model, and a rhyming model, and the recent work of @xcite and @xcite . our approach distinguishes itself in using a general-purpose pretrained language model, but more importantly in its use of human-curated constraints and templates. these allow for generating high-quality poems with a very simple approach. ## methodology the general idea of our approach is to take a pretrained language model (in this case gpt-2) and apply hard constraints to the generation procedure so that it can only output text satisfying various poetic constraints. these constraints can be broadly divided into hard constraints (e.g., number of syllables) and soft constraints (e.g., sounding poetic), and our methodology can be separated similarly. our generation process is in figure 2 . ## experiments we used human input to test our sonnets against both model-generated and human-written sonnets. to test adherence to a theme throughout a son- expert evaluation category mean p-value poetryme grammar 4.50* 1.71×10 -4 emotion 4.30* 3.13×10 -3 poetic 4.30* 3.13×10 -3 human 4.10* 5.77×10 -3 theme 2.60 0.211286 benhardt et al. grammar 3.83* 0.03 emotion 3.67* 0.05 poetic 3.75* 0.04 human 3.75* 0.02 theme 2.42 0.06 human-written poems grammar 1.36 1.00×10 -6 emotion 1.4 5.00×10 -6 poetic 1.64 5.40×10 -5 human 1.36 1.00×10 -6 theme 1.57 7.70×10 -5 figure 3: starred figures indicate average scores @xmath0, and underlined figures indicate that the p-value is low enough (< 0.05) to claim that this higher average is statistically significant. net, we desired baselines that generate whole sonnets with user-provided themes. this limited our competitors, as some generate discrete quatrains or generate without input themes (e.g., deepspeare), leaving only benhardt et al. (2018) and poetryme (oliveira, 2012); see section a.2. furthermore, an evaluation of poetry quality is incomplete without human-written sonnets, selected from sonnets.org. though these poems do not have an explicit theme, we selected poems that followed our five themes. to optimally test our model, we conducted an internal analysis and selected k values sampled from 3, 5, or 7, b values sampled from 3, 5, or 7, and repetition penalty values sampled from 1.4, 1.6, or 1.8 that we concluded produced the highest quality sonnets. to evaluate adherence to theme, we generated poems with themes "death," "darkness," "forest," "love," and "wisdom." in each test, respondents compared six randomly selected pairs of sonnets, with each of our sonnets displayed with a competing model/human-written sonnet generated with the same theme word. respondents indicated which of the two sonnets performed better in categories of theme, poeticness, grammar, emotion, and likelihood of being humanwritten. detailed instructions are in a.3. ## conclusion we propose a novel method for generating highquality poems that uses pos templating to determine a logical syntactical structure and rigorously maintains constraints necessary for any sonnet. our method is highly versatile, with poetic factors like alliteration, internal rhyme, repetition, and theme adjustable to ensure creative output. after extensive surveys conducted with expert evaluators and mturk participants, our model's success over similar competitors is evident, though our model's poems, like those of most computer poetry generators, remain distinguishable from human written poems. while we were unable to compare our model's performance to that of chatgpt, our finetuned gpt-2 requires far less computing power than subsequent gpt models. additionally, while we commenced this project's evaluation prior to the release of chatgpt, after a preliminary qualitative evaluation, chatgpt seems to produce very generic poetry (see a.7). thus, for this particular application, our model may be a viable method that is more cost-effective and produces relatively high-quality sonnets.