SlowGuess/NAACL_2022_OCR huggingface dataset:
We provide detailed instructions and ask them to evaluate the each poem on a scale from 1 (not at all) to 5 (very) on the following criteria: 1) Discourse Coherence: whether the sonnet is well organized, with the sentences smoothly connected and flow together logically and aesthetically, 2) Originality/Creativity: the usage of original ideas in the poem, including imagination, rhetorical devices, etc., 3) Poetic in language: how well the poem adopts descriptive and vivid language that often has an economical or condensed usage, 4) Emotion Evoking: if the poem is emotionally abundant and make the readers emphasize with the writer. At last, we ask the annotators to judge if the sonnet is written by a poet with serious goals to write a poem. In total, we evaluate 50 sonnets for each baseline and the gold standard (human) model. Each sonnet is rated by three professionals.
The average inter-annotator agreement (IAA) in terms of Pearson correlation is 0.61 with p-value  $< 0.01$ , meaning that our collected ratings are highly reliable. We also conduct paired t-test for significance testing. The difference between our best performing model and the best baseline is significant. Considering the expertise required, human evaluators are paid $25 per hour.
Table 2 summarizes the format checking and novelty scores of our model compared to the baselines. We can see that human poets tend to incorporate more variations and do not strictly follow the meter and rhyme constraints, which computers are good at. GPT-3 fails to learn the sonnet formats through massive pretraining and few-shot learning despite its gigantic size. Prosaic falls short of meter-checking because is only trained to generate rhyming verses. Since we utilize the phonetics
DC	O	P	E	WH
Hafez	3.09	3.01	3.05	2.95	41.3%
Few-shot GPT3	3.43	3.10	2.86	3.11	52.7%
Prosaic	3.25	2.95	2.97	2.98	46.0%
Zest w/o fig	3.57*	3.25	3.35	3.13	58.7%
Zest	3.52	3.41*	3.66*	3.22*	62.0%*
Human	3.82	3.54	3.68	3.56	83.3%
Table 3: Expert ratings on several criteria to assess sonnet quality: discourse-level coherence (DC), originality/creativity (O), poeticness in language (P), emotion evoking (E), and written by human (WH). We show average scores with 1 denoting the worst and 5 the best. We boldface/underline the best/second best scores. * denotes that paired t-test shows that our model variations (Zest w/o fig, and Zest) outperform the best baseline in all aspects with statistical significance (p-value  $< 0.05$ ).
information provided in the CMU dictionary, Zest achieves  $100\%$  success in rhyme words pairing. As for novelty, Zest generates most diversely and is best at that arousing mental pictures of concrete objects among machines.
5.2 Results of Human Evaluation
Table 3 presents the performance of the aforementioned evaluation criteria: coherence, originality, poeticness, and emotion-evoking. Our models (Zest w/o fig, and Zest) outperform the baselines in all aspects by a large margin.
Comparison between our own models. Compared with Prosaic which also generates poems from non-poetic texts, our models generate more coherent sonnets with great statistical significance (p-value  $< 0.01$ ), showing the superiority of explicit sketch planning over generating from scratch (i.e., end-to-end generation).
Zest  $w/o$  fig generates more coherently than Zest (p-value  $< 0.10$ ). However, Zest achieves high scores in originality, poeticness by a large margin (+0.2). Hence, we still consider it as our best model. It is also noteworthy that Zest is the most emotion-evoking system among all machines even though we do not have explicit sentiment control. Poem theories have shown that emotion appeals lie in the following aspects: the general topic, the word choice, vivid descriptions, figurative language, insights and experience (Scheub, 2002). We posit that aesthetic features in the Zest arouse emotion appeals.
Analysis for high poeticness. Zest is on par with humans in terms of poeticness score, meaning that our models generate highly descriptive, vivid,
and condensed text. With manual examination, we attribute such high poeticness to three aspects. First, the imagery and similes clearly represents traits of poems. Second, in keyword-planning we ensure that at least three concepts will be presented per line, and thus the generation module naturally become economical in word usage to include all the information. Lastly, with the constraint decoding algorithm to insert keywords, we inevitably become less natural (e.g., miss conjunctions and auxiliary verbs). While this can be a drawback in other generation tasks, the occasional omission of such auxiliary words is just opportune for sonnets, and adds to the flavor of a poem. The examples in table 4 helps demonstrate these points.
We conduct case study to better understand the advantages of our model over the baselines. Table 4 lists the generated sonnets by Hafez, Prosaic and Zest given the same title: "The Four Seasons".
Problems with the Baselines Hafez chooses words that are related to the title as rhyme words. However, topically related rhyme words are not sufficient for overall coherence. While it is locally understandable, the sonnet generated by Hafez is divergent and disconnected when sentences are put together. On the other hand, Prosaic mimics the rhyme and topical properties of poems, but still generate highly prosaic and colloquial sentences that are not poetic at all.
Advantages of Our Model Thanks to content planning, Zest w/o fig generates a well-organized sonnet that describes the four seasons from winter to autumn in a logical order. Despite minor grammar errors, the full model Zest benefits from vivid descriptions and natural imagery such as 'whispers rumors of a winter coming', 'blossom of the season', and 'sudden like a flash'.
6.2 Impact of Keywords
By comparing Zest w/o fig versus Prosaic, our human evaluation results already show that content planning contributes to discourse-level coherence. In addition, we provide the keywords along with the sonnet generated by Zest, and ask human annotators to judge if the sonnet can be condensed into those keywords. Results are shown in Figure 4. We observe that  $82\%$  of the time the planed
Hafez	Score	Prosaic	Score
Delightful hours over long ago,
Succeed to hang the little hook and light.
The darker still and more extended snow,
The fresh invention bore the better night.	C: 2.7	The leaves fall and the air is cooler,
Were we known with seasons? Spring is coming,
Of course there isn't that much humor,
And it is now blooming.	C: 3.3
Another party started getting heavy,
And never had a little bit of summer.
And got a trigger on the finger really,
Or something going by the name of winter.	O: 3.0	Even when everyone else's warm,
Someone would want to know what we could do in may,
else we didn't known - It was the first storm,
It's how they used to do every other day.	O: 2.7
Retreat the colors and fun in afternoon
And bought the tickets loved a royal song
A sacred person join and ride in moon	P: 3.0	What aren't you looking up Right behind us, no clouds,
Just know no one's will ever see the clouds or thunder,
And nobody other than the crowds,
Before now, we had just started to wonder.	P: 3.0
Away the author go and winding long,	E: 3.0	We all aren't sure what we can keep,
they couldn't see me, like they were about to reap.	E: 3.3
Oh maybe get a little out of sight,
And wounded victim suffered from the site.
Zest w/o fig	Score	Zest	Score
Falling snow, I feel a little cooler,
Is a sign of stormy winter coming,
Meet the honest friends and gather humor,
Happy spring and just admire the blooming.	C: 3.7	The snow is falling, whether getting cooler,
Whispers rumors of a winter coming,
Gather honest, peace and love and humor,
Blossom of the happy season blooming.	C: 3.7
The heat and ocean air becoming warm,
Little birds and beautiful songs in May,
Leaves and flowers probably just the storm,
Among the lunar moons and summer day.	O: 3.3	Wind gets heat and the air becoming warm,
The birds enjoy the little lovely may,
Beneath the leaves, flowers survive the storm,
The moon is shining on a summer day.	O: 4.0
Sudden rain and downpour from the thunder,
And summer always fill hotels with crowds,
Take a shower and give the spring a wonder,
Watch the blue sky and far behind the clouds.	P: 3.3	Sudden like a flash comes rain with thunder,
The summer vibes fill the running crowds,
Because of shower, spring became a wonder,
The sky is high and blue like sea with clouds.	P: 4.0
In months the future vegetables reap,
The years and seasons never really keep.	The coming months are watching future reap,
Those years and seasons bring us all to keep.	E: 3.3
Table 4: An example of the generated sonnets from four systems with the same title: "The Four Seasons". The scores are average numbers of three human ratings on the following criteria: coherence (C), originality (O), poetic in language (P), and emotion evokingness (E). We underline the planed keywords and highlight the figurative languages in blue.
Figure 4: Pie chart showing whether the generated sonnet be condensed into the planed keywords.
keywords successfully guide the generation by providing salient points of the sonnet. We then conduct error analysis on the rest  $18\%$ . Top two reasons among the fail cases are: 1) the decoding step generates novel contents that are not represented by the keywords  $(8\%)$ , and 2) the polishing step alters the original meaning of planed keywords  $(6\%)$ .
6.3 Limitation and Future Direction
Sonnets are divided in to multiple stanzas. Lines within a stanza are more interlaced than across stanza, and the start of a new one usually indi
cates transition to another viewpoint. Our current approach could not capture such structural characteristics during planning and generation, and we hope to investigate these features in future work.
We also plan to extend this poem generation pipeline to other languages. For example, pretrained LMs (e.g. multilingual T5) and existing rhyming resources (r.g. rhymes.woxikon.com provides rhymes in 13 languages) already made the first and second component transferable to other languages.
Poetry Generation Automatic poetry generation before the deep learning age relies heavily on templates, norms, or rule-based approaches (Gervás, 2001; Manurung, 2004; Manurung et al., 2012). Neural approaches to automatic poetry generation pay little attention to the coherence issue of long poems. For example, Wang et al. (2016); Lau et al. (2018); Yi et al. (2018); Liu et al. (2019)
merely target at generating the first stanza (four lines) of a poem. For longer poems such as sonnets, Ghazvininejad et al. (2016) propose to use related words as rhyme words to achieve topical relatedness, and later propose to generate discourse-level coherent English sonnets by French-English translation (Ghazvininejad et al., 2018). Van de Cruys (2020) propose a naive RNN framework to generate rhyming verses from prosaic texts by imposing a priori word probability constraints. We on the other hand achieve discourse-level coherence by learning from standard, non-poetic texts.
Other related works to boost the creativity of generated poems include adding rhetorical (Liu et al., 2019) and influence factors (e.g., historical background) as latent variables (Yi et al., 2020). To the best of our knowledge, we are the first to explore adding both figurative speeches and meter-and-rhyme constraints to poetry generation without relying on poetry data.
Content Planning Content planning for automatic text generation originates in the 1970s (Meehan, 1977). Recently, the plan-and-write generation framework has shown to be efficient in creative content generation (Wang et al., 2016; Martin et al., 2018; Peng et al., 2018; Yao et al., 2019; Gao et al., 2019; Goldfarb-Tarrant et al., 2019). The framework employs a hierarchical paradigm and helps to produce more coherent and controllable generation than generating from scratch (Fan et al., 2019; Goldfarb-Tarrant et al., 2020). However, all existing works under this line learn the storyline/plot from the target domain for improved coherence. We on the other hand adopt content planning to disentangle the training from the decoding step which aims at circumventing the shortage of sizable creative contents for training supervised models.
We investigate the possibility of generating sonnets without training on poems at all. We propose a hierarchical planning-based framework to generate sonnets which first plans the high-level content of the poem, refine the predicted keywords by adding poetic features, and then achieve decoding-time control to impose the meter-and-rhyme constraints. Extensive automatic and expert evaluation show that our model can generate sonnets that use rich imagery and are globally coherent, poetic, and emotion provoking.
The authors would like to thank the members of PLUSLab and the anonymous reviewers for helpful comments. This work is supported in part by the DARPA Machine Common Sense (MCS) program under Cooperative Agreement N66001-19-2-4032. Yufei Tian is supported by an Amazon Fellowship.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. Comet: Commonsense transformers for automatic knowledge graph construction. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL).
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS 2020).
Tuhin Chakrabarty, Smaranda Muresan, and Nanyun Peng. 2020. Generating similes effortlessly like a pro: A style transfer approach for simile generation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Angela Fan, Mike Lewis, and Yann Dauphin. 2019. Strategies for structuring story generation. In ACL.
Jun Gao, Wei Bi, Xiaojiang Liu, Junhui Li, Guodong Zhou, and Shuming Shi. 2019. A discrete cvae for response generation on short-text conversation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
Pablo Gervás. 2001. An expert system for the composition of formal Spanish poetry. In Applications and Innovations in Intelligent Systems VIII, pages 19-32. Springer.
Marjan Ghazvininejad, Yejin Choi, and Kevin Knight. 2018. Neural poetry translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 67-71.
Marjan Ghazvininejad, Xing Shi, Yejin Choi, and Kevin Knight. 2016. Generating topical poetry. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1183-1191.
Raymond W Gibbs Jr, Raymond W Gibbs, and Jr Gibbs. 1994. The poetics of mind: Figurative thought, language, and understanding. Cambridge University Press.
Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. 2020. Content planning for neural story generation with aristotelian rescoring. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng. 2019. Plan, write, and revise: an interactive system for open-domain story generation. In 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2019), Demonstrations Track, volume 4, pages 89-97.
Michael Hart. 2004. Project Gutenberg.
Karl Moritz Hermann, Tomás Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In NIPS, pages 1693-1701.
Chris Hokamp and Qun Liu. 2017. Lexically constrained decoding for sequence generation using grid beam search. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.
Justine Kao and Dan Jurafsky. 2012. A computational analysis of style, affect, and imagery in contemporary poetry. In Proceedings of the NAACL-HLT 2012 workshop on computational linguistics for literature, pages 8-17.
Jey Han Lau, Trevor Cohn, Timothy Baldwin, Julian Brooke, and Adam Hammond. 2018. Deep-spare: A joint neural model of poetic language, meter and rhyme. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
Hugo Liu and Push Singh. 2004. Conceptnet—a practical commonsense reasoning tool-kit. BT technology journal, 22(4):211-226.
Zhiqiang Liu, Zuohui Fu, Jie Cao, Gerard de Melo, Yik-Cheung Tam, Cheng Niu, and Jie Zhou. 2019. Rhetorically controlled encoder-decoder for modern chinese poetry generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1992-2001.
Hisar Manurung. 2004. An evolutionary algorithm approach to poetry generation.
Ruli Manurung, Graeme Ritchie, and Henry Thompson. 2012. Using genetic algorithms to create meaningful poetic text. Journal of Experimental & Theoretical Artificial Intelligence, 24(1):43-64.
Lara J Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O Riedl. 2018. Event representations for automated story generation with deep neural nets. In Thirty-Second AAAI Conference on Artificial Intelligence.
James R Meehan. 1977. Tale-spin, an interactive program that writes stories. In *IJCAI*, volume 77, page 9198.
Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight. 2018. Towards controllable story generation. In *NAACL Workshop*.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research (JMLR).
Stuart Rose, Dave Engel, Nick Cramer, and Wendy Cowley. 2010. Automatic keyword extraction from individual documents. Text mining: applications and theory, 1:1-20.
Harold Scheub. 2002. The Poem in the Story: Music, Poetry, and Narrative. Univ of Wisconsin Press.
Michael S Silk. 2006. Interaction in poetic imagery: with special reference to early Greek poetry. Cambridge University Press.
Yulia Tsvetkov, Leonid Boytsov, Anatole Gershman, Eric Nyberg, and Chris Dyer. 2014. Metaphor detection with cross-lingual model transfer. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 248-258.
Tim Van de Cruys. 2020. Automatic poetry generation from prosaic text. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2471-2480.
Zhe Wang, Wei He, Hua Wu, Haiyang Wu, Wei Li, Haifeng Wang, and Enhong Chen. 2016. Chinese poetry generation with planning based neural network. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers.
Robert L Weide. 1998. The cmu pronouncing dictionary. URL: http://www.speech.cs.cmu.edu/cgi-bin/cmudict.
Yunyi Yang, Yunhao Li, and Xiaojun Quan. 2021. Ubar: Towards fully end-to-end task-oriented dialog systems with gpt-2. Proceedings of the AAAI Conference on Artificial Intelligence.
Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7378-7385.
Xiaoyuan Yi, Ruoyu Li, Cheng Yang, Wenhao Li, and Maosong Sun. 2020. Mixpoet: Diverse poetry generation via learning controllable mixed latent space. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9450-9457.
Xiaoyuan Yi, Maosong Sun, Ruoyu Li, and Wenhao Li. 2018. Automatic poetry generation with mutual reinforcement learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3143-3153.