Sentimental Poetry Generation (2020)

Abstract: The paper investigates how well poetry can be generated to contain a specific sentiment, and whether readers of the poetry experience the intended sentiment. The poetry generator consists of a bi-directional Long Short-Term Memory (LSTM) model, combined with rhyme pair generation, rule-based word prediction methods, and tree search for extending generation possibilities. The LSTM network was trained on a set of English poetry written and published by users on a public website. Human judges evaluated poems generated by the system, both with a positive and negative sentiment. The results indicate that while there are some weaknesses in the system compared to other state-of-the-art solutions, it is fully capable of generating poetry with an inherent sentiment that is perceived by readers.

## introduction poetry generation is a type of linguistic creativity that requires certain qualities in both form and content, as well as the creation of understandable, meaningful and poetic language. a central part of poetry is the experience of the reader, including the emotions poetry can evoke. the overarching goal of this work is to explore and develop methods for generating poetry with a specific (pre-defined) inherent sentiment, which can be experienced by the readers. earlier approaches to poetry generation followed a range of paths, such as methods based on templates @xcite or corpora @xcite , evolutionary @xcite or case-based reasoning @xcite approaches, and generate-and-test @xcite or blackboard @xcite architectures. however, in recent years deep learners have proven powerful as poetry generators, including systems combining neural models with other techniques. as described below, long short-term memory (lstm, @xcite networks are the most used solutions in state-of-the-art systems, so an lstm was implemented here, with an experimental focus on which specific network architecture and parameter settings would produce the best poetry word prediction model. the sentiment of the poems to be generated was decided in advance, with human judges rating their quality and how well the sentiment was perceived. unlike the one-pass generation for previous neural networks models, @xcite proposed a generative model with a polishing schema, refining the rnn-generated poem through several iterations. also, while previous models were based on maximum likelihood estimation (mle), which optimizes word-level loss and can lead to the systems remembering common patterns of the training corpus, @xcite added reinforcement learning to a basic generator pre-trained with mle, simultaneously training two generators that learn both from the teacher (rewarder) and from each other. automatic rewarders were designed corresponding to four criteria: fluency, coherence, meaningfulness, and overall quality. @xcite aimed to generate poetry in the style of a specific author, using an lstm to predict the next word based on a previous word sequence, with the embeddings of the document currently being analysed used to support the model at every step. two datasets were used to train the model, in english and russian. several of the systems presented above implement a form of user input to influence the mood of the poetry, often related to a given sentiment. how-ever, two such systems are particularly important in the way they include the use of sentiment: in the corpus-based approach of full-face poetry generation @xcite , the system decides on a mood by checking the average sentiment of a set of newspaper articles posted during the previous 24 hours, and then selects one of the five articles with the highest resp. lowest sentiment value. @xcite system includes an emotional personality aspect implementing both sentiment analysis and emotional modelling. to extract sentiment, the sentistrength @xcite tool is used, rating positive and negative scores on a valence value scale. an average arousal value of the input is calculated using affective norms for english words (anew, @xcite , and the poem's emotional state is set by combining valence and arousal. wordnet-affect @xcite is used to build a hierarchy of words describing emotional states in order to generate the affective content of poems. ## data set the data set used in the experiments here is the english part of the data collected by @xcite . it consists of poems written and published by users on a public website, which leads to a variance in the quality of content, but both the large size and variance in content are positive factors for network training. the provided data set was already cleaned, with all types of punctuation removed and all letters converted to lowercase. however, there were some inconsistencies in how contractions were represented, with some appearing in a joined form (e.g., wouldve) and others as separate words (would ve). hence all spaces between regular contractions were removed, as were line break markers (<@xmath0, with every individual poem was instead represented by single individual lines, so that the structure of the generated poems would not be constrained. the original data set contains 3, 943, 982 poems and 155, 066, 504 tokens, with a vocabulary of 708, 727 unique tokens. most of the unique tokens come from misspellings, alternative spellings, irregular words and names. the training data was shortened to specifically reduce the vocabulary size, in order to remove words that very rarely appear in the data set, and to reduce the size of the matrix representation of data used in training the network. to reduce the data set, tests were run on vocabulary sizes of 10, 000 -30, 000, finding how many of the poems only included words within a given vocabulary, as can be seen in an important aspect of the vocabulary is the inclusion of words with sentiment values, since they would be generated to add sentiment to the poetry. using vader @xcite , the data sets were investigated for how many unique words they contained with a polar (non-neutral) sentiment, i.e., either having a positive or a negative sentiment. those are also reported in due to the increased number of unique sentiment words with increased vocabulary sizes, but also due to memory restrictions and larger vocabularies resulting in greater training times, the vocabulary sizes for neural network training was chosen to be 10, 000 and 20, 000. as to evaluate the trained networks, new test data was created from the original data sets, in the same the test data sets are also shown in the training of the neural network is done by creating input sequences to be fed through the network, but also creating the correct output which is then compared to the output of the network. the input is therefore created by choosing a sequence of the training data with a given length, and the token following that sequence as the correct output. a training data sequence length of 5 was chosen, both for memory storage reasons and since 5 was decided to be the shortest possible line length of the generated poetry. resticting the sequence length will also make the network predict the next words based on just a short sequence, instead of all of the poem that is already generated, which could lead to more variation. before creating the network input matrices, every poem was reversed, with the last word being the first, and so on, following the approach used by @xcite . this is done so it is possible to start with the ending rhyme word of a line of poetry, and generate the rest of the line backwards from that rhyme-word. ## architecture this section introduces the architecture for the system implemented in this project. the first part describes the long short-term memory network that was implemented, while the second part describes the complete poetry generation process. ## network experiments experiment were conducted to decide the final architecture of the lstm network, and what parameters to use in the implementation of the poetry generation system. the network training was performed on a tesla p100-pcie-16gb gpu. the learning rate was initially set to 0.9 for the training of all networks. a monitor was implemented on the validation loss calculated on validation samples each epoch, reducing the learning rate after a given period when the validation loss has not decreased. the period before reducing the learning rate was set to 5 epochs, and the reduction of the learning rate to a factor of 0.3. the minimum limit for the learning rate was set to 0.001. the dropout probability rate was 0.6 for all non-recurrent units. this value was chosen based on @xcite , where the dropout rate used for medium lstm (650 units per layer) was 0.5, and 0.65 for large lstm (1500 units per layer). two different data sets were used in the experiments and the parameters tested were the numbers of hidden layers, units in each hidden layer, and training epochs. the test data input was fed through the networks, and predictions were measured against true values, using three measures: ## evaluating the generated poetry human judges evaluated the generated poetry both in itself and with regard to the intended sentiment. the poetry was rated along the dimensions suggested by manurung (2004): grammaticality, meaningfulness, and poeticness, on a 1-3 scale (1 being "not", 2 "partially, and 3 "fully"). for the sentiment rating, the human judges evaluated the poetry using three categories: negative sentiment, no sentiment (neutral), and positive sentiment. if a poem was evaluated as having negative or positive sentiment, it was graded with a score of 1-3 ("slightly", "quite", and "very" negative/positive). twenty poems were evaluated (i.e., 40 stanzas and 160 lines), generated to contain 10 each with positive and negative sentiment. thirty human judges participated, evaluating a selection of 6 or 7 poems each, with a near-equal amount of positive and negative sentiments. the results show that on average the majority of judges perceived the poems to contain the sentiment value intended by the system, but the degrees of the sentiment value, both for negative and positive poems, are rated considerably lower by the human judges than the ratings given by vader. one reason for this is the very high ratings that vader assigns, both for negative and positive sentiment scores, with the generator discouraging the use of words with an opposite sentiment value (compared to the intended value), while encouraging the use of words with a "correct" sentiment value, given by vader. the degree ratings from the human judges on the other hand reflect that a lack of the words with an opposite sentiment value does not result in a high degree of sentiment being perceived. it is also interesting to note the consistency of the degree ratings given by the human judges, where positive and negative sentiment poems are on average rated with almost the same degree of sentiment. three of the generated poems are presented below: #8 with one of the lowest scores in the experiment results, and #3 and #14 with some of the highest. along with each poem are its scores, with the first array presenting the grammaticality, meaningfulness and poeticness scores. the second array shows the percentage of human judges who found the poem to contain negative, neutral, or positive sentiment. poem 8 achieved a score of 1.22 for both grammaticality and meaningfulness, while poem 14 achieved a score of 1.73 for those. the poeticness scores for both poem 8 and 14 are below average, while poem 3 achieved the highest poeticness score of all the poems with 2.5, and also high scores for grammaticality and meaningfulness. a common trait among all the generated poems is incorrect use of articles and conjunctions, in addition to erroneous use of other word classes and poor sentence structure. examples of this are the sequences so most from an till and amid to our so most from currently in poem 8. another noticeable aspect is the rhyme pairs in the poetry not always rhyming, e.g., exceptions and solutions at the end of lines 6 and 8 in poem 14. this is due to the system pairing rhyme words based on the last three syllables using cmudict. a similar aspect is the consistent lack of repetitiveness found in the poetry, since repeated use of words is highly discouraged by the system, to avoid constant use of the same high predictions words, resulting in the poems lacking an often used poetic technique, to consciously ## conclusion and future work a system capable of generating poetry with inherent sentiment has been designed and implemented, with the main system component being a bi-directional long short-term memory network used for generating word predictions based on a given input sequence. the network was trained on a data set consisting of poetry written by humans. other components of the final generation system are algorithms and rule-based methods for influencing word predictions and word choices during the generation process, and a search algorithm for expanding the possibilities of generated sequences. the implemented system was used to generate 20 poems in total, all consisting of two stanzas with four lines each. 10 of the poems were generated to contain an inherent positive sentiment value, while the other 10 were generated to contain a negative sentiment value. several experiments were conducted, both regarding the lstm network and on the generated poetry. the first experiment was on training different lstm networks with varying architecture details, with the goal of training the best performing network model to use in the implementation of the final poetry generation system. two other experiments were conducted on the final generated poetry, both involving human judges evaluating the generated poetry. the first of these experiment consisted of the judges evaluating the poetry based on three standard evaluation criteria. this enabled evaluation of the performance of the poetry generation system, and comparison to other works that have been conducted in this field. in the second experiment the human judges evaluated the sentiment value they perceived generated poetry to contain, in order to investigate whether the system was capable of generating poetry with an inherent sentiment value that would be perceived as intended by human readers. the results of the experiments varied, with lstm experiments giving word perplexity scores worse than state-of-the-art solutions. applying some standard evaluation metrics showed one of the metrics achieving similar values to state-of-theart solutions, while the other metric gave poorer results, one reason being the influence from the subpar prediction performance of the lstm network. the experiment for evaluating the sentiment of the generated poetry produced good results. while there is a lack of similar experiments by others to compare to, the results show a clear trend of the human judges perceiving the poetry to contain the intended sentiment value. possible future work could include implementing additional features or other architectures, such as word embedding models or language models like bert @xcite that show prominent results for text analysis, or use mutual reinforcement, which has given state-of-the-art results in poetry generation @xcite . the data set used to train the neural network model has a considerable effect on the system's performance and the generated poetry. hence using different data sets, especially data containing poetry of a generally accepted higher quality, would probably improve the system. adding only perfect rhymes for rhyme pair generation, or a strict poetic form based on syllables, such as the sonnet form, could improve the poetic qualities of the output. the main feature of the generation system is to generate poetry with an inherent sentiment, and this can also be further developed. first, the system needs to generate poetry with a wider range of sentiment value words, as it current only uses words with neutral sentiment or sentiment values corresponding to the intended sentiment. adding more words with opposite sentiment could increase the poetry's emotional dynamic. the sentiment feature could also be extended to generate poetry with a wider range of emotions, e.g., by using emotional modelling similarly to @xcite , or by adapting the system to generate poetry with an inherent degree of a specific sentiment, not just a general negative or positive value.