Poetry Generation Combining Poetry Theme Labels Representations (2023)

Abstract: Ancient Chinese poetry is the earliest literary genre that took shape in Chinese literature and has a dissemination effect, showing China’s profound cultural heritage. At the same time, the generation of ancient poetry is an important task in the field of digital humanities, which is of great significance to the inheritance of national culture and the education of ancient poetry. The current work in the field of poetry generation is mainly aimed at improving the fluency and structural accuracy of words and sentences, ignoring the theme unity of poetry generation results. In order to solve this problem, this paper proposes a graph neural network poetry theme representation model based on label embedding. On the basis of the network representation of poetry, the topic feature representation of poetry is constructed and learned from the granularity of words. Then, the features of the poetry theme representation model are combined with the autoregressive language model to construct a theme-oriented ancient Chinese poetry generation model TLPG (Poetry Generation with Theme Label). Through machine evaluation and evaluation by experts in related fields, the model proposed in this paper has significantly improved the topic consistency of poetry generation compared with existing work on the premise of ensuring the fluency and format accuracy of poetry.

## introduction ancient chinese poetry is a short, vivid form of literary genre, and the depictions in ancient poetry are a living reflection of the daily life of the chinese ancestors as well as their inner prayers and expectations. in world literature, ancient chinese poetry is also an important means of demonstrating the power of the chinese language. it is interesting to note that although automated machine poetry compositions fall short of human beings in terms of rhyme, mood, and feeling @xcite . however automated poetry generation is still worth researching. on the one hand, machine-generated ancient poems can assist students in generating a deeper understanding of poetry @xcite . on the other hand, the emergence of multimodal technologies has made poetry learning not only limited to texts @xcite a variety of information can be utilized in the study of poetry teaching and learning. in terms of poetic format, poems of different genres and eras often have different formats (e.g., poetic style, rhyme, etc.), which involve grammatical, semantic, and phonological aspects. current research tends to establish grammatical analysis as well as symbolic representation of phonology for ancient poems, which makes poetry often contain explicit thematic information @xcite . themes in poetry are implicit textual information are important references for poetry appreciation and creation, so this paper focuses on the representation and integration of theme features in the process of poetry generation. at present, the mainstream poetry generation models are mainly based on the transformer @xcite and generate poems in the form of autoregressive language models. however, unlike general text generation tasks, poetry is a highly structured literary genre @xcite . in the poetry generation task, the input part of the model is designed with a special identifier to learn the specific format of the poem @xcite . during the learning process this special identifier learns potential formatting information in a particular type of poetry. in the poetry generation stage, such a special identifier functions accordingly in the decoding stage. making it possible to generate text in a well-formed poetry genre. this paper focuses on proposing a label embedding-based graph neural network poetry topic representation model, while combining it with applications on poetry generation tasks. thus, the main research work in this paper is as follows: ## poetry generation related work research on automatic poetry generation started in the 1960s, and early poetry generation studies used a system architecture for generating lyrics given a melody, selecting words from a word list library, then testing and filtering them, repicking them if they are not suitable, and moving on to the next position if they are @xcite . and then some scholars added genetic algorithms to the poetry generation task, and @xcite designed a coding method based on flat and oblique, introduced the coding information into the fitness function, and added elitism and roulette algorithms in genetic algorithms, but the high computational complexity of genetic algorithms and the possibility of sometimes falling into local optimal solutions made the quality of poetry generation uneven. with the advent of statistical machine learning, @xcite used the vector space model to formulate the relationships between sentences as vectors, and then used cosine similarity to compare the relationships between sentence pairs and extract relevant statements from blog posts for generation. @xcite used information retrieval techniques to retrieve a set of poems related to keywords from a database, and sub-phrasing, and then applied abstraction techniques to generate poems using the sub-phrasing results. however, the above method does not constrain the poetry topic and format, and the generation is not effective. in order to better incorporate control attributes into the generation process, advanced model structures have been gradually proposed as neural networks and deep learning techniques have gradually matured. @xcite first introduced deep neural networks into the poetry generation task and proposed a model combining recurrent neural networks (rnn) for ancient chinese poetry generation. @xcite proposed the use of a variational auto-encoder (vae) with a framework overall attribute discriminator to achieve the control of generation direction by combining vae with a discriminator. yi et al (2020 ) used a semi-supervised vae framework to generate poems with more thematic and semantic richness considering the style of the poems as a combination of multiple factors. to improve the thematic relevance of poetry generation, some scholars have proposed using a working memory model that utilizes an internal memory to store and access multiple subject terms. @xcite used an unsupervised approach to enhance the diversity of poetry generation by maximizing the mutual information between the style distribution and the output distribution. zhang (2020) et al. based on the transformer structure and incorporated input identifiers to make information about the formatting and meter of the poem displayed by participating in model training to generate poems with a more standardized format. the generation model in this paper aims to ensure the quality of poetry generation while generating poems with a more uniform theme, improve the interactivity with users, and be applied in the system construction. ## experimental results and analysis the experimental results are shown in the results also demonstrate that tlpg improves thematic consistency while also ensuring the regularity of its generated format. when using the gtp model for top-k sampling, it is easy to lose control over the regularity of the generated poems due to its random nature, and significantly lower than the other models in terms of format accuracy, tone accuracy and rhyme accuracy, and perplexity and theme consistency. meanwhile, this paper conducted ablation experiments on the model, and the results of these three indicators are not discussed here because there is no variable control on format, tone and rhyme aspects, and the final results of the ablation experiments are shown in with the overall experimental framework structure unchanged, there is a decrease in theme consistency, which also indicates that adding the vector obtained by training the theme representation again after establishing the connection with the verse to the generative model is more effective than simply using the lda model output vector representation of the theme labels for input. ## conclusion in this paper, we propose a method for constructing a graph network of poetry texts, based on which a tag embedding method is combined with the graph network to obtain a poetry theme style modeling and representation, which is finally applied to the task of generating ancient chinese poetry on a specific theme. the model is firstly based on the existing autoregressive language model framework and constructs a poetry generation model tlpg incorporating poetry theme tags by setting specific identifiers in the input layer, the model in this paper can ensure that the generated poems are formatted as required, and at the same time, combined with the poetry theme representation, can guide the model to generate poems that are more in line with the user's desired theme style. after the evaluation of real datasets and the results of manual evaluation by experts in related fields, the method proposed in this paper can not only improve the accuracy and quality of poetry generation, but also meet the personalized needs of users. in addition, this paper will further investigate the introduction of more types of texts such as song lyrics and yuan songs into the tlpg model to improve the generalization ability and application scenarios of the model. in the future, with the continuous development and improvement of natural language processing technology, the ancient poetry generation technology will also be more widely and deeply applied.