Venue: ACL 2020

# Automatic Poetry Generation from Prosaic Text

Tim Van de Cruys

Institut de Recherche en Informatique de Toulouse (IRIT)

Artificial and Natural Intelligence Toulouse Institute (ANITH)

CNRS, Toulouse

tim.vandecruys@irit.fr

###### Abstract

In the last few years, a number of successful approaches have emerged that are able to adequately model various aspects of natural language. In particular, language models based on neural networks have improved the state of the art with regard to predictive language modeling, while topic models are successful at capturing clear-cut, semantic dimensions. In this paper, we explore how these approaches can be adapted and combined to model the linguistic and literary aspects needed for poetry generation. The system is exclusively trained on standard, non-poetic text, and its output is constrained in order to confer a poetic character to the generated verse. The framework is applied to the generation of poems in both English and French, and is equally evaluated for both languages. Even though it only uses standard, non-poetic text as input, the system yields state of the art results for poetry generation.

## 1 Introduction

Automatic poetry generation is a challenging task for a computational system. For a poem to be meaningful, both linguistic and literary aspects need to be taken into account. First of all, a poetry generation system needs to properly model language phenomena, such as syntactic well-formedness and topical coherence. Furthermore, the system needs to incorporate various constraints (such as form and rhyme) that are related to a particular poetic genre. And finally, the system needs to exhibit a certain amount of literary creativity, which makes the poem interesting and worthwhile to read.

In recent years, a number of fruitful nlp approaches have emerged that are able to adequately model various aspects of natural language. In particular, neural network language models have improved the state of the art in language modeling, while topic models are successful at capturing clear-cut, semantic dimensions. In this paper, we explore how these approaches can be adapted and combined in order to model both the linguistic and literary aspects that are required for poetry generation. More specifically, we make use of recurrent neural networks in an encoder-decoder configuration. The encoder first constructs a representation of an entire sentence by sequentially incorporating each word of the sentence into a fixed-size hidden state vector. The final representation is then given to the decoder, which emits a sequence of words according to a probability distribution derived from the hidden state of the input sentence. By training the network to predict the next sentence with the current sentence as input, the network learns to generate plain text with a certain discourse coherence. By modifying the probability distribution yielded by the decoder, we enforce the incorporation of poetic constraints, such that the network can be exploited for the generation of poetic verse. It is important to note that the poetry system is not trained on poetic texts; rather, the system is trained on a corpus of standard, prosaic texts extracted from the web, and it will be the constraints applied to the network's probability distribution that confer a poetic character to the generated verse.

The rest of this article is structured as follows. In section 2, we present an overview of related work on automatic poetry generation. Section 3 describes the different components of our model. In section 4, we present an extensive human evaluation of our model, as well as a number of examples generated by the system. Section 5, then, concludes and discusses some future research directions.

## 2 Related work

Early computational implementations that go beyond mere mechanical creativity have often relied on rule-based or template-based methods. One of the first examples is the ASPERA system [6]

2001) for Spanish, which relies on a complex knowledge base, a set of rules, and case-based reasoning. Other approaches include Manurung et al. (2012), which combines rule-based generation with genetic algorithms, Goncalo Oliveira (2012)'s PoeTryMe generation system, which relies on chart generation and various optimization strategies, and Veale (2013), which exploits metaphorical expressions using a pattern-based approach.

Whereas poetry generation with rule-based and template-based models has an inherent tendency to be somewhat rigid in structure, advances in statistical methods for language generation have opened up new avenues for a more varied and heterogeneous approach to creative language generation. Greene et al. (2010), for example, use an \(n\)-gram language model in combination with a rhythmic model implemented with finite-state transducers. And more recently, recurrent neural networks (rnns) have been exploited for poetry generation; Zhang and Lapata (2014) use an encoder-decoder rnn for Chinese poetry generation, in which one rnn builds up a hidden representation of the current line in a poem, and another rnn predicts the next line word by word, based on the hidden representation of the current line. The system is trained on a corpus of Chinese poems. Yan (2016) tries to improve upon the encoder-decoder approach by incorporating a method of iterative improvement: the network constructs a candidate poem in each iteration, and the representation of the former iteration is used in the creation of the next one. And Wang et al. (2016) extend the method using an attention mechanism.

Ghazvininejad et al. (2016) combine rnns (for syntactic fluency) with distributional similarity (for the modeling of semantic coherence) and finite state automata (for imposing literary constraints such as meter and rhyme). Their system, _Hafez_, is able to produce well-formed poems with a reasonable degree of semantic coherence, based on a user-defined topic. Hopkins and Kiela (2017) focus on rhythmic verse; they combine an rnn, trained on a phonetic representation of poems, with a cascade of weighted finite state transducers. Lau et al. (2018) present a joint neural network model for the generation of sonnets, called _Deep-spear_, that incorporates the training of rhyme and rhythm into the neural network; the network learns iambic stress patterns from data, while rhyming word pairs are separated from non-rhyming ones using a margin-based loss. And a number of recent papers extend neural poetry generation for Chinese with various improvements, such as unsupervised style disentanglement Yang et al. (2018), reinforcement learning Yi et al. (2018), and rhetorical control Liu et al. (2019).

Note that all existing statistical models are trained on or otherwise make use of a corpus of poetry; to our knowledge, our system is the first to generate poetry with a model that is exclusively trained on a generic corpus, which means the poetic character is endowed by the model itself. Secondly, we make use of a latent semantic model in order to model topical coherence, which is equally novel.

## 3 Model

### Neural architecture

The core of the poetry system is a neural network architecture, trained to predict the next sentence \(S_{i+1}\) given the current sentence \(S_{i}\). The architecture is made up of gated recurrent units (grus; Cho et al., 2014) that are linked together in an encoder-decoder setup. The encoder sequentially reads in each word \(w^{i}_{1,...,N}\) of sentence \(S_{i}\) (represented by its word embedding \(\mathbf{x}\)) such that, at each time step \(t_{i}\), a hidden state \(\hat{\mathbf{h}}_{t}\) is computed based on the current word's embedding \(\mathbf{x}_{t}\) and the previous time step's hidden state \(\hat{\mathbf{h}}_{t-1}\). For each time step, the hidden state \(\hat{\mathbf{h}}_{t}\) is computed according to the following equations:

\[\mathbf{r}_{t}= \sigma(\mathbf{W}_{r}\mathbf{x}_{t}+\mathbf{U}_{r}\hat{\mathbf{ h}}_{t-1}) \tag{1}\] \[\mathbf{z}_{t}= \sigma(\mathbf{W}_{z}\mathbf{x}_{t}+\mathbf{U}_{z}\hat{\mathbf{ h}}_{t-1})\] (2) \[\bar{\mathbf{h}}_{t}= \tanh(\mathbf{W}\mathbf{x}_{t}+\mathbf{U}(\mathbf{r}_{t}\odot \hat{\mathbf{h}}_{t-1}))\] (3) \[\hat{\mathbf{h}}_{t}= (1-\mathbf{z}_{t})\odot\hat{\mathbf{h}}_{t-1}+\mathbf{z}_{t} \odot\bar{\mathbf{h}}_{t} \tag{4}\]

where \(\mathbf{r}_{t}\) represents the gru's reset gate, \(\mathbf{z}_{t}\) represents the update gate, \(\bar{\mathbf{h}}_{t}\) represents the candidate update state, and \(\odot\) represents pointwise multiplication.

\(\hat{\mathbf{h}}_{t}\) can be interpreted as a representation of the sequence \(w_{1},\ldots,w_{t}\), and the final hidden state \(\hat{\mathbf{h}}_{N}\) will therefore be a representation of the entire sentence. This final hidden encoder state is transferred to the decoder. The decoder then sequentially predicts the next sentence word by word, conditioned on the encoder's final hidden representation; at each time step \(t_{i+1}\), the decoder equally computes a hidden state \(\mathbf{h}_{t}\) based on the current word's embedding \(\mathbf{x}_{t}\) (which was predicted by the decoder 

in the previous time step) and the previous time step's hidden state \({\bf h}_{t-1}\) (the first hidden state of the decoder is initialized by \(\hat{\mathbf{h}}_{N}\) and the first word is a symbolic start token). The computations for each time step \({\bf h}_{t}\) of the decoder are equal to the ones used in the encoder (equations 1 to 4).

In order to fully exploit the entire sequence of representations yielded by the encoder, we augment the base architecture with an attention mechanism, known as _general_ attention Luong et al. (2015). The attention mechanism allows the decoder to consult the entire set of hidden states computed by the encoder; at each time-step--for the generation of each word in sentence \(S_{i+1}\)--the decoder determines which words in sentence \(S_{i}\) are relevant, and accordingly selects a linear combination of the entire set of hidden states. In order to do so, we first compute an attention vector \({\bf a}_{t}\), which attributes a weight to each hidden state \(\hat{\mathbf{h}}_{i}\) yielded by the encoder (based on the decoder's current hidden state \({\bf h}_{t}\)). according to equation 5:

\[{\bf a}_{t}(i)=\frac{\exp(\mathrm{score}({\bf h}_{t},\hat{\mathbf{h}}_{i}))}{ \sum_{i^{\prime}}\exp(\mathrm{score}({\bf h}_{t},\hat{\mathbf{h}}_{i^{\prime }}))} \tag{5}\]

where

\[\mathrm{score}({\bf h}_{t},\hat{\mathbf{h}}_{i})={\bf h}_{t}^{T}{\bf W}_{a} \hat{\mathbf{h}}_{i} \tag{6}\]

The next step is to compute a global context vector \({\bf c}_{t}\), which is a weighted average (based on attention vector \({\bf a}_{t}\)) of all of the encoder's hidden states. The resulting context vector is then combined with the original decoder hidden state in order to compute a new, attention-enhanced hidden state \(\tilde{\mathbf{h}}_{t}\).

\[\tilde{\mathbf{h}}_{t}=\tanh({\bf W}_{c}[{\bf c}_{t};{\bf h}_{t}]) \tag{7}\]

where \([:,:]\) represents vector concatenation. Finally, this resulting hidden state \(\tilde{\mathbf{h}}_{t}\) is transformed into a probability distribution \(p({\bf w}^{t}|w^{<t},S_{i})\) over the entire vocabulary using a softmax layer.

\[p({\bf w}^{t}|w^{<t},S_{i})=\mathrm{softmax}({\bf W}_{\textbf{s}}\tilde{ \mathbf{h}}_{t}) \tag{8}\]

As an objective function, the sum of the log-probabilities of the next sentence is optimized, conditioned on the hidden state representation of the current sentence.

\[J_{t}=\sum_{(S_{i},S_{i+1})\in C}-\log p(S_{i}|S_{i+1}) \tag{9}\]

At inference time, for the generation of a verse, each word is then sampled randomly according to the output probability distribution. Crucially, the decoder is trained to predict the next sentence in reverse, such that the last word of the verse is the first one that is generated. This reverse operation is important for an effective incorporation of rhyme, as will be explained in the next section. A graphical representation of the architecture, which includes the constraints discussed below, is given in Figure 1.

Figure 1: Graphical representation of the poetry generation model. The encoder encodes the current verse, and the final representation is given to the decoder, which predicts the next verse word by word in reverse. The attention mechanism is represented for the first time step. The rhyme prior is applied to the first time step, and the topic prior is optionally applied to all time steps, mediated by the entropy threshold of the network’s output distribution.

 



### Poetic constraints as _a priori_ distributions

As the neural architecture described above is trained on generic text, its output will in no way resemble poetic verse. In order to endow the generated output with a certain poetic character, we modify the neural network's output probability distribution through the application of a prior probability distribution, that constrains the standard output probability distribution, and boosts the probability of words that are a good fit within the defined constraints. We will consider two kinds of constraints: a rhyme constraint and a topical constraint.

#### 3.2.1 Rhyme constraint

In order to adequately model the rhyme constraint, we make use of a phonetic representation of words, extracted from the online dictionary _Wiktionary_.1 For each word of the vocabulary, we determine its rhyme sound (i.e. the final group of vowels, optionally followed by a group of consonants), as well as the group of consonants that precedes the group of vowels. A sample of rhymes that are thus extracted is represented in Table 1.

Footnote 1: www.wiktionary.org

The next step then consists in creating a probability distribution for a particular rhyme sound that we want the verse to adhere to:

\[p_{rhyme}(\mathbf{w})=\frac{1}{Z}\mathbf{x}\text{ with }\begin{cases}x_{ i}=1&\text{if $i\in R$}\\ x_{i}=\epsilon&\text{otherwise}\end{cases} \tag{10}\]

where \(R\) is the set of words that contain the required rhyme sound, \(\epsilon\) is a small value close to zero, used for numerical stability, and \(Z\) is a normalization factor in order to ensure a probability distribution. We can now use \(p_{rhyme}(\mathbf{w})\) as a prior probability distribution in order to reweight the neural network's standard output probability distribution--according to Equation 11--each time the rhyme scheme demands it:

\[p_{out}(\mathbf{w})=\frac{1}{Z}(p(\mathbf{w^{t}}|w^{<t},S_{i})\odot p_{rhyme} (\mathbf{w})) \tag{11}\]

where \(\odot\) represents pointwise multiplication.2 As we noted before, each verse is generated in reverse; the reweighting of rhyme words is applied at the first step of the decoding process, and the rhyme word is generated first. This prevents the generation of an ill-chosen rhyme word that does not fit well with the rest of the verse.

Footnote 2: Such a multiplicative combination of probability distributions is also known as a Product of Experts (Hinton, 2002).

#### 3.2.2 Topical constraint

For the modeling of topical coherence, we make use of a latent semantic model based on non-negative matrix factorization (nmf; Lee & Seung, 2001). Previous research has shown that non-negative factorization methods are able to induce clear-cut, interpretable topical dimensions Murphy et al. (2012). As input to the method, we construct a frequency matrix **A**, which captures co-occurrence frequencies of vocabulary words and context words.3 This matrix is then factorized into two non-negative matrices **W** and **H**,

Footnote 3: The raw frequencies are weighted using _pointwise mutual information_Turney and Pantel (2010).

\[\textbf{A}_{i\times j}\approx\textbf{W}_{i\times k}\textbf{H}_{k\times j} \tag{12}\]

where \(k\) is much smaller than \(i,j\) so that both instances and features are expressed in terms of a few components. Non-negative matrix factorization enforces the constraint that all three matrices must be non-negative, so all elements must be greater than or equal to zero. Using the minimization of the Kullback-Leibler divergence as an objective function, we want to find the matrices **W** and **H** for which the divergence between **A** and **WH** (the multiplication of **W** and **H**) is the smallest. The factorization is carried out through the iterative application of update rules. Matrices **W** and **H** are randomly initialized, and the rules in 13 and 14 are iteratively applied--alternating between them. In each iteration, each vector is adequately normalized, so that all dimension values sum to 1.

\[\textbf{H}_{a\mu}\leftarrow\textbf{H}_{a\mu}\frac{\sum_{i}\textbf{ W}_{ia}\frac{\textbf{A}_{i\mu}}{(\textbf{W}\textbf{H})_{j\mu}}}{\sum_{k} \textbf{W}_{ka}} \tag{13}\] \[\textbf{W}_{ia}\leftarrow\textbf{W}_{ia}\frac{\sum_{\mu}\textbf{ H}_{a\mu}\frac{\textbf{A}_{i\mu}}{(\textbf{W}\textbf{H})_{j\mu}}}{\sum_{v} \textbf{H}_{av}} \tag{14}\]

\begin{table}
\begin{tabular}{l c} \hline \hline word & rhyme \\ \hline embrace & (mb1, es) \\ suitcase & (tk, es) \\ sacrifice & (f, ans) \\ paradise & (d, ans) \\ \hline reproduit & (dt, i) \\ thérapie & (p, i) \\ examen & (m, \(\bar{\epsilon}\)) \\ canalien & (dj, \(\bar{\epsilon}\)) \\ \hline \hline \end{tabular}
\end{table}
Table 1: A number of rhyme examples extracted from _Wiktionary_, for both English and French.



Tables 2 and 3 present a number of example dimensions induced by the model, for both English and French.

The factorization that comes out of the nmf model can be interpreted probabilistically [15, 4]: matrix **W** can be considered as \(p({\bf w}|k)\), i.e. the probability of a word given a latent dimension \(k\). In order to constrain the network's output to a certain topic, it would be straightforward to simply use \(p({\bf w}|k)\) as another prior probability distribution applied to each output. Initial experiments, however, indicated that such a blind modification of the output probability distribution for every word of the output sequence is detrimental to syntactic fluency. In order to combine syntactic fluency with topical consistency, we therefore condition the weighting of the output probability distribution on the entropy of that distribution: when the output distribution's entropy is low, the neural network is certain of its choice for the next word in order to generate a well-formed sentence, so we will not change it. On the other hand, when the entropy is high, we will modify the distribution by using the topical distribution \(p({\bf w}|k)\) for a particular latent dimension as prior probability distribution--analogous to Equation 11--in order to inject the desired topic. The entropy threshold, above which the modified distribution is used, is set experimentally.

Note that the rhyme constraint and the topical constraint can straightforwardly be combined in order to generate a topical rhyme word, through pairwise multiplication of the three relevant distributions, and subsequent normalization in order to ensure a probability distribution.

### A global optimization framework

The generation of a verse is embedded within a global optimization framework. There are two reasons to integrate the generation of a verse within an optimization procedure. First of all, the generation of a verse is a sampling process, which is subject to chance. The optimization framework allows us to choose the best sample according to the constraints presented above. Secondly, the optimization allows us to define a number of additional criteria, that assist in the selection of the best verse. For each final verse, the model generates a considerable number of candidates; each candidate verse is then scored according to the following criteria:

* the log-probability score of the generated verse, according to the encoder-decoder architecture (section 3.1);
* compliance with the rhyme constraint (section 3.2.1); additionally, the extraction of the preceding group of consonants (cf. Table 1) allows us to give a higher score to rhyme words with disparate preceding consonant groups, which elicits more interesting rhymes;
* compliance with the topical constraint (section 3.2.2); the score is modeled as the sum of the probabilities of all words for the defined dimension;
* the optimal number of syllables, modeled as a Gaussian distribution with mean \(\mu\) and standard deviation \(\sigma\);4 Footnote 4: We equally experimented with rhythmic constraints based on meter and stress, but initial experiments indicated that the system had a tendency to output very rigid verse. Simple syllable counting tends to yield more interesting variation.
* the log-probability score of a standard _n_-gram model.

The score for each criterion is normalized to the interval \([0,1]\) using _min-max_ normalization, and the harmonic mean of all scores is taken as the final score for each candidate.5 After generation of a predefined number of candidates, we keep the candidate with the highest score, and append it to the poem.

Footnote 5: The harmonic mean is computed as \(\frac{n}{\sum^{i=1}_{m=1}\frac{1}{x}}\); we choose this measure in order to balance the different scores.

\begin{table}
\begin{tabular}{c c c} \hline dim 13 & dim 22 & dim 28 \\ \hline sorrow & railway & planets \\ longing & trains & planet \\ admiration & rail & cosmic \\ earnest & station & universe \\ \hline \end{tabular}
\end{table}
Table 2: Three example dimensions from the nmf model for English (4 words with highest probability)

\begin{table}
\begin{tabular}{l l l} \hline dim 1 & dim 20 & dim 25 \\ \hline tendresse & gare & hypocrisie \\ joie & bus & mensonge \\ bonheur & metro & accuser \\ sourires & rer & hypocrite \\ \hline \end{tabular}
\end{table}
Table 3: Three example dimensions from the nmf model for French (4 words with highest probability) 

Results and evaluation

### Implementational details

We train two different models for the generation of poetry in both English and French. The neural architecture is trained on a large corpus of generic web texts, constructed on the basis of the CommonCrawl corpus.6 In order to filter out noise and retain clean, orderly training data, we apply the following filtering steps:

Footnote 6: commoncrawl.org

* we only keep sentences written in the relevant language;
* we only keep sentences of up to 20 words;
* we only keep sentences that contain at least one function word from a predefined list--the idea again is to filter out noisy sentences, and only keep well-formed, grammatical ones; we create a list of about 10 highly frequent function words, specific to each language;
* of all the sentences that remain after these filtering steps, we only keep the ones that appear successively within a document.

Using the filtering steps laid out above, we construct a training corpus of 500 million words for each language. We employ a vocabulary of 15k words (those with highest frequency throughout the corpus); less frequent words are replaced by an <unk> token, the probability of which is set to zero during generation.

Both encoder and decoder are made up of two gru layers with a hidden state of size 2048, and the word embeddings are of size 512. Encoder, decoder, and output embeddings are all shared [37]. Model parameters are optimized using stochastic gradient descent with an initial learning rate of 0.2, which is divided by 4 when the loss does no longer improve on a held-out validation set. We use a batch size of 64, and we apply gradient clipping. The neural architecture has been implemented using PyTorch [29], with substantial reliance on the OpenNMT module [10]. For the application of the topical constraint, we use an entropy threshold of \(2.70\).

The _n_-gram model is a standard Kneser-Ney smoothed trigram model implemented using _KenLM_[1], and the nmf model is factorized to 100 dimensions. Both the _n_-gram model and the nmf model are trained on a large, 10 billion word corpus, equally constructed from web texts without any filtering steps except for language identification. For syllable length, we use \(\mu=12,\sigma=2\).

We generate about 2000 candidates for each verse, according to a fixed rhyme scheme (ABAB CDCD). Note that no human selection whatsoever has been applied to the poems used in the evaluation; all poems have been generated in a single run, without _cherry picking_ the best examples. Four representative examples of poems generated by the system are given in Figure 2.

### Evaluation procedure

Quantitatively evaluating creativity is far from straightforward, and this is no less true for creative artefacts that are automatically generated. Automatic evaluation measures that compute the overlap of system output with gold reference texts (such as bleu or Rouge), and which might be used for the evaluation of standard generation tasks, are of little use when it comes to creative language generation. The majority of research into creative language generation therefore makes use of some form of human evaluation, even though one needs to keep in mind that the evaluation of textual creativity is an inherently subjective task, especially with regard to poetic value. For a discussion of the subject, see Goncalo Oliveira (2017).

We adopt the evaluation framework by Zhang and Lapata (2014), in which human annotators are asked to evaluate poems on a five point scale with regard to a number of characteristics, viz.

* _fluency_: is the poem grammatical and syntactically well-formed?
* _coherence_: is the poem thematically structured?
* _meaningfulness_: does the poem convey a meaningful message to the reader?
* _poeticness_: does the text display the features of a poem?

Additionally, we ask annotators to judge if the poem is written by a human or a computer.

In total, we evaluate four different sets of poems, yielded by different model instantiations. The different sets of poems considered during evaluation 

* _rnn_: poems generated by the neural architecture defined in section 3.1, without any added constraints;
* _rhyme_: poems generated by the neural architecture, augmented with the rhyme constraint;
* _nmf\({}_{rand}\)_: poems generated by the neural architecture, augmented with both the rhyme constraint and the topical constraint, where one of the automatically induced nmf dimensions is selected randomly;
* _nmf\({}_{spec}\)_: poems generated by the neural architecture, augmented with both the rhyme constraint and the topical constraint, where one of the automatically induced nmf dimensions is specified manually.7 Footnote 7: This can be regarded as manually defining the theme of the generated poem. The specified dimension is selected for its poetic character.

For a proper comparison of our system, we equally include:

* _random_: poems yielded by a baseline model where, for each verse, we select a random sentence (that contains between 7 and 15 words) from a large corpus; the idea is that the lines selected by the baseline model should be fairly fluent (as they come from an actual corpus), but lacking in coherence (due to their random selection);
* _human_: poems written by human poets; the scores on this set of poems function as an upper bound;
* _Hafez_ and _Deep-speare_: poems generated by two state of the art poetry generation systems for English, respectively by Ghazvininejad et al. (2016) and Lau et al. (2018); we use the code made available by the respective authors.8 Note that we only compare to other poetry generation systems for English, as no other readily available systems exist for French.

Footnote 8: _Hafez_ needs to be initialized with user-defined topics; for a fair comparison, we seed the system with the top words of the nmf dimension used for our best performing model.

### Results for English

For English, 22 annotators evaluated 40 poems in total (5 poems for each of the different sets considered in the evaluation; each poem was evaluated by at least 4 annotators). The annotators consist of native speakers of English, as well as master students in English linguistics and literature. For the _human_ set, we select five poems by well-established English poets that follow the same rhyme scheme as the generated ones.9 For _nmf\({}_{spec}\)_, we select dimension 13 of Table 2. The results of the evaluation for English are presented in the upper part of Table 4.

Footnote 9: The selected poets are W.H. Auden, E.E. Cummings, Philip Larkin, Sarojini Naidu, and Sylvia Plath.

First of all, note that all our model instantiations score better than th

Figure 2: Four representative examples of poems generated by the system; the left-hand poems, in English, are respectively generated using dimensions 13 and 28 (cf. Table 2); the right-hand poems, in French, are generated using dimensions 1 and 25 (cf. Table 3).

 

even with regard to grammatical fluency. The good scores on fluency for the constrained models indicate that the applied constraints do not disrupt the grammaticality of the generated verse (_rhyme_ is significantly different10 with \(p<0.05\); _nmf\({}_{rand}\)_ and _nmf\({}_{spec}\)_ with \(p<0.01\); recall that the baseline consists of actual sentences from a corpus). Secondly, we note that the rhyme constraint seems to improve poeticness (though not significantly), while the topical constraint seems to improve both coherence (\(p<0.01\) for _nmf\({}_{spec}\)_) and meaningfulness (not significantly). Interestingly, a large proportion of the poems produced by the _rhyme_ model are labeled as human, even though the other scores are fairly low. The score for poeticness is considerably higher (\(p<0.01\)) for _nmf\({}_{spec}\)_ (with a manually specified theme selected for its poeticness) than for _nnf\({}_{rand}\)_ (with a randomly selected topic, which will often be more mundane). And the best scores on all criteria are obtained with the _nmf\({}_{spec}\)_ model, for which more than half of the poems are judged to be written by a human; moreover, the difference between _nmf\({}_{spec}\)_ and human poetry is not significant. Finally, our poetry generation compares favourably to previous work: _nmf\({}_{spec}\)_ scores markedly and significantly better than _Deep-spare_ (which does not differ significantly from the random baseline), and it equally attains better scores than _Hafez_ on all four criteria (though not significantly so).

Footnote 10: Significance testing is carried out using a two-tailed permutation test.

### Results for French

The setup of the French evaluation is analogous to the English one: an equal number of 22 annotators have evaluated a total of 30 poems (5 poems for each of the six sets considered in the evaluation; each poem was evaluated by at least 4 annotators). The annotators are all native speakers of French. For the human poems, we select five poems with the same rhyme scheme as the generated ones, among the highest ranked ones on short-edition.com--a website with submissions by amateur poets. For _nmf\({}_{spec}\)_, we select dimension 1 of Table 3. The results for French are presented in the lower part of Table 4.

Generally speaking, we see that the results for French confirm those for English. First of all, all model instantiations obtain better scores than the _random_ baseline model, even with regard to fluency (\(p<0.01\)), again confirming that the application of the rhyme constraint and topical constraint are not detrimental to the grammaticality of the verse. Secondly, the rhyme constraint significantly improves the score for poeticness (\(p<0.05\) compared to _rnn_), while the topical constraint improves both coherence (\(p<0.05\)) and meaningfulness (\(p<0.01\)). Contrary to the English results, only a small proportion of poems from the _rhyme_ model are thought to be human. We do again see that the score for poeticness is considerably higher (\(p<0.01\)) for

\begin{table}
\begin{tabular}{l r r r r r} \hline \hline  & \multicolumn{6}{c}{English} \\ \cline{2-7} model & fluency & coherence & meaningfulness & poeticness & written by human (\%) \\ \hline _rnn_ & 2.95 & 2.50 & 2.45 & 2.55 & 0.18 \\ _thyme_ & 3.41 & 2.77 & 2.82 & 2.95 & **0.59** \\ _nmf\({}_{rand}\)_ & 3.32 & 3.09 & 2.86 & 2.95 & 0.32 \\ _nmf\({}_{spec}\)_ & **3.64** & **3.41** & **3.27** & **3.86** & 0.55 \\ \hline _random_ & 2.68 & 2.09 & 1.91 & 2.41 & 0.14 \\ _Deep-spare_ & 2.11 & 2.00 & 2.00 & 3.00 & 0.22 \\ _Hafez_ & 3.44 & 3.11 & 3.11 & 3.50 & 0.53 \\ _human_ & 3.73 & 3.73 & 3.68 & 4.00 & 0.73 \\ \hline \hline \end{tabular} 
\begin{tabular}{l r r r r r} \hline \hline  & \multicolumn{6}{c}{French} \\ \cline{2-7} model & fluency & coherence & meaningfulness & poeticness & written by human (\%) \\ \hline _rnn_ & 3.45 & 2.73 & 2.59 & 2.55 & 0.27 \\ _rhyrme_ & **3.82** & 2.55 & 2.18 & 3.23 & 0.14 \\ _nmf\({}_{rand}\)_ & 3.64 & 3.32 & 3.09 & 2.86 & 0.27 \\ _nmf\({}_{spec}\)_ & **3.82** & **3.82** & **3.55** & **3.95** & **0.45** \\ \hline _random_ & 2.95 & 1.86 & 1.68 & 2.18 & 0.00 \\ _human_ & 4.59 & 4.59 & 4.50 & 4.81 & 0.95 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results of the human evaluation (mean score of all annotators) for English and French; values in **bold** indicate best performance of all generation models 

\(nmf_{spec}\) than for \(nmf_{rand}\), which seems to indicate that the topic of a poem is an important factor in people's judgements on poeticness. Finally, we again see that the best scores on all criteria are obtained with \(nmf_{spec}\), for which almost half of the poems are judged to be written by a human.

## 5 Conclusion

We presented a system for automatic poetry generation that is trained exclusively on standard, non-poetic text. The system uses a recurrent neural encoder-decoder architecture in order to generate candidate verses, incorporating poetic and topical constraints by modifying the output probability distribution of the neural network. The best verse is then selected for inclusion in the poem, using a global optimization framework. We trained the system on both English and French, and equally carried out a human evaluation for both languages. The results indicate that the system is able to generate credible poetry, that scores well with regard to fluency and coherence, as well as meaningfulness and poeicness. Compared to previous systems, our model achieves state of the art performance, even though it is trained on standard, non-poetic text. In our best setup, about half of the generated poems are judged to be written by a human.

We conclude with a number of future research avenues. First of all, we would like to experiment with different neural network architectures. Specifically, we believe hierarchical approaches Serban et al. (2017) as well as the Transformer network Vaswani et al. (2017) would be particularly suitable to poetry generation. Secondly, we would like to incorporate further poetic devices, especially those based on meaning. Gripping poetry often relies on figurative language use, such as symbolism and metaphor. A successful incorporation of such devices would mean a significant step towards truly inspired poetry generation. And finally, we would like to adapt the model for automatic poetry translation--as we feel that the constraint-based approach lends itself perfectly to a poetry translation model that is able to adhere to an original poem in both form and meaning.

In order to facilitate reproduction of the results and encourage further research, the poetry generation system is made available as open source software. The current version can be downloaded at [https://github.com/timvdc/poetry](https://github.com/timvdc/poetry).

## Acknowledgements

This work is supported by a grant overseen by the French National Research Agency anr (project quantum - ANR-19-CE23-0025); it has equally benefited from a GPU donated by nvidia Corporation.

## References

* Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1724-1734. Association for Computational Linguistics.
* Ding et al. (2008) Chris Ding, Tao Li, and Wei Peng. 2008. On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. _Computational Statistics & Data Analysis_, 52(8):3913-3927.
* Gaussier and Goutte (2005) Eric Gaussier and Cyril Goutte. 2005. Relation between plsa and nmf and implications. In _Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval_, pages 601-602. ACM.
* Gervas (2001) Pablo Gervas. 2001. An expert system for the composition of formal spanish poetry. In _Applications and Innovations in Intelligent Systems VIII_, pages 19-32, London. Springer.
* Ghazvininejad et al. (2016) Marjan Ghazvininejad, Xing Shi, Yejin Choi, and Kevin Knight. 2016. Generating topical poetry. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 1183-1191, Austin, Texas. Association for Computational Linguistics.
* Oliveira (2012) Hugo Goncalao Oliveira. 2012. Poetryme: a versatile platform for poetry generation. _Computational Creativity, Concept Invention, and General Intelligence_, 1:21.
* Oliveira (2017) Hugo Goncalao Oliveira. 2017. A survey on intelligent poetry generation: Languages, features, techniques, reutilisation and evaluation. In _Proceedings of the 10th International Conference on Natural Language Generation_, pages 11-20.
* Greene et al. (2010) Erica Greene, Tugba Bodrumlu, and Kevin Knight. 2010. Automatic analysis of rhythmic poetry with applications to generation and translation. In _Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing_, pages 524-533. Association for Computational Linguistics.
* Heafield (2011) Kenneth Heafield. 2011. KenLM: faster and smaller language model queries. In _Proceedings of the 

EMNLP 2011 Sixth Workshop on Statistical Machine Translation_, pages 187-197, Edinburgh, Scotland, United Kingdom.
* Hinton (2002) Geoffrey E Hinton. 2002. Training products of experts by minimizing contrastive divergence. _Neural computation_, 14(8):1771-1800.
* Hopkins and Kiela (2017) Jack Hopkins and Douwe Kiela. 2017. Automatically generating rhythmic verse with neural networks. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 168-178. Association for Computational Linguistics.
* Klein et al. (2017) Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. Opennmt: Opensource toolkit for neural machine translation. In _Proceedings of ACL 2017, System Demonstrations_, pages 67-72. Association for Computational Linguistics.
* Lau et al. (2018) Jey Han Lau, Trevor Cohn, Timothy Baldwin, Julian Brooke, and Adam Hammond. 2018. Deep-speare: A joint neural model of poetic language, meter and rhyme. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1948-1958. Association for Computational Linguistics.
* Lee and Seung (2001) Daniel D Lee and H Sebastian Seung. 2001. Algorithms for non-negative matrix factorization. In _Advances in neural information processing systems_, pages 556-562.
* Liu et al. (2019) Zhiqiang Liu, Zuohui Fu, Jie Cao, Gerard de Melo, Yik-Cheung Tam, Cheng Niu, and Jie Zhou. 2019. Rhetorically controlled encoder-decoder for modern Chinese poetry generation. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 1992-2001, Florence, Italy. Association for Computational Linguistics.
* Luong et al. (2015) Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_, pages 1412-1421. Association for Computational Linguistics.
* Manurung et al. (2012) Ruli Manurung, Graeme Ritchie, and Henry Thompson. 2012. Using genetic algorithms to create meaningful poetic text. _Journal of Experimental & Theoretical Artificial Intelligence_, 24(1):43-64.
* Murphy et al. (2012) Brian Murphy, Partha Talukdar, and Tom Mitchell. 2012. Learning effective and interpretable semantic models using non-negative sparse embedding. In _Proceedings of COLING 2012_, pages 1933-1950. The COLING 2012 Organizing Committee.
* Paszke et al. (2017) Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. In _Advances in Neural Information Processing Systems_.
* Woff (2017) Ofir Press and Licor Wolf. 2017. Using the output embedding to improve language models. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_, pages 157-163. Association for Computational Linguistics.
* Serban et al. (2017) Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In _Thirty-First AAAI Conference on Artificial Intelligence_.
* Turney and Pantel (2010) Peter D Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. _Journal of artificial intelligence research_, 37:141-188.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Advances in Neural Information Processing Systems_, pages 5998-6008.
* Veale (2013) Tony Veale. 2013. Less rhyme, more reason: Knowledge-based poetry generation with feeling, insight and wit. In _Proceedings of the international conference on computational creativity_, pages 152-159.
* Wang et al. (2016) Qixin Wang, Tianyi Luo, Dong Wang, and Chao Xing. 2016. Chinese song iambics generation with neural attention-based model. In _Proceedings of International Joint Conference on Artificial Intelligence_, pages 2943-2949.
* Yan (2016) Rui Yan. 2016. i, poet: Automatic poetry composition through recurrent neural networks with iterative polishing schema. In _Proceedings of International Joint Conference on Artificial Intelligence_, pages 2238-2244.
* Yang et al. (2018) Cheng Yang, Maosong Sun, Xiaoyuan Yi, and Wenhao Li. 2018. Stylistic Chinese poetry generation via unsupervised style disentanglement. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 3960-3969, Brussels, Belgium. Association for Computational Linguistics.
* Yi et al. (2018) Xiaoyuan Yi, Maosong Sun, Ruoyu Li, and Wenhao Li. 2018. Automatic poetry generation with mutual reinforcement learning. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 3143-3153, Brussels, Belgium. Association for Computational Linguistics.
* Zhang and Lapata (2014) Xingxing Zhang and Mirella Lapata. 2014. Chinese poetry generation with recurrent neural networks. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 670-680. Association for Computational Linguistics.