Composing Ci with Reinforced Non-autoregressive Text Generation (2022)

Abstract: Composing Ci (also widely known as Song Ci), a special type of classical Chinese poetry, requires to follow particular format once their tune patterns are given. To automatically generate a well-formed Ci, text generation systems should strictly take into account pre-defined rigid formats (e.g., length and rhyme). Yet, most existing approaches regard Ci generation as a conventional sequence-to-sequence task and use autoregressive models, while it is challenging for such models to properly handle the constraints (according to tune patterns) of Ci during the generation process. Moreover, consider that with the format prepared, Ci generation can be operated by an efficient synchronous process, where autoregressive models are limited in doing so since they follow the character-by-character generation protocol. Therefore, in this paper, we propose to compose Ci through a non-autoregressive approach, which not only ensure that the generation process accommodates tune patterns by controlling the rhythm and essential meaning of each sentence, but also allow the model to perform synchronous generation. In addition, we further improve our approach by applying reinforcement learning to the generation process with the rigid constraints of Ci as well as the diversity in content serving as rewards, so as to further maintain the format and content requirement. Experiments on a collected Ci dataset confirm that our proposed approach outperforms strong baselines and previous studies in terms of both automatic evaluation metrics and human judgements.

## introduction ci is a special type of chinese poetry and different from general poems, where their shape (including lengths and tunes) is determined by tune patterns (i.e., 词牌, also known as brand names) defining figure 1 : the first half of an example ci following the tune pattern "念奴娇" foot_2 (charm of a singer), where all essential words in this ci are highlighted in blue. translations for each clause are provided for reference. 4 the particular syllable and rhythm schemes that a ci should stick to. 2 over the past one thousand years, composing ci has long been an interesting game and continued to present days for chinese people to demonstrate one's literary and artistic accomplishments. given that ci composition is challenging, to perform this task with artificial intelligence is meaningful since it is a good test for controlled natural language generation under specific constraints (i.e., the length and rhyme of ci should follow the constraints given by the tune pattern). recently, text generation models based on deep neural networks (e.g., lstm @xcite and transformer @xcite ) have been widely used for poem and ci generation @xcite @xcite @xcite and demonstrated their validity in doing so. among previous studies, most mainly follow the conventional text generation paradigm which performs an autoregressive generation process by generating a poem or ci in a character-by-character manner. however, since ci is usually longer than classical poem, these autoregressive models face the challenge of losing semantic coherence in-between the beginning and end of a ci when it is too long and thus might lead to inferior results because a high-quality ci always requires good coherence in the topic. to illustrate, figure 1 shows a well-known ci with the tune pattern "念奴娇" (charm of a singer), where every sentence in this ci are correlated to each other and stick to the main topic. therefore, ci shows the following two characteristics which are required to be addressed carefully during its generation process: (1) once the tune pattern is given, the rigid format of the ci (i.e., the length and rhyme) is determined; (2) different parts in a ci should show high relevance in semantics (e.g., stick to a particular topic). these characteristics suggest that non-autoregressive models (whose effectiveness for text generation has already been demonstrated in machine translation @xcite , image captioning @xcite , and summarization @xcite ) have their potential to be appropriate choices for this task. moreover, another advantage of non-autoregressive models is that they are able to generate different parts of a ci synchronously, which is more efficient compared with autoregressive ones. still, although nonautoregressive models show aforementioned superiority, there are further improvements needed for ci composition, such as guiding the model to generate clauses strictly following the length and rhyme requirements of the tune pattern and providing more diversified generation results, which are normally hard to be controlled through conventional supervised/unsupervised functions. consider that reinforcement learning (rl) is able to guide the parameter optimization process of a model through object-oriented rewards and it has been demonstrated to be effective in many natural language generation tasks such as dialogue generation @xcite , paraphrase generation @xcite , and image captioning @xcite , it is expected to be also effective in helping the nonautoregressive models to generate ci that better follows the constraints of the tune pattern. in this paper, we propose, cigen, a nonautoregressive model for ci generation with a given tune pattern, where we employ a key word guided generation process to firstly generate essential (ess) words (e.g., the ones highlighted in blue in figure 1 ) that convey the important meaning for each part (e.g., sentence) and then generate the final full ci. to further enhance the non-autoregressive model, we apply rl to ci composition so as to accommodate the formats and rhyming constraints, which are generally hard to learn by the conventional supervised or unsupervised learning methods since it is not easy to design normal loss functions for them. in evaluation, we test our approach on a collected song ci dataset foot_4 , where our approach outperforms strong baselines and previous studies on both automatic and human evaluation metrics. ## the proposed approach figure 2 illustrates the overall architecture of our cigen for ci composition with the given tune pattern z, where the format (denoted as @xmath0, • • • , x t , • • • , x t with x t presenting the format of the t-th character and t the number of characters) of ci is obtained based on the tune pattern z and then used to generate the intermediate ess words (denoted as k), and the candidate ci (denoted as @xmath1, • • • , v t , • • • , v t ), then the generated ess words and the candidate ci are then combined to obtain the final ci (denoted as @xmath2, • • • , y t , • • • , y t ). therefore, the process of the proposed non-autoregressive approach for ci composition is formally expressed by ## related work chinese ci generation is generally considered as one type of chinese archaic style text generation tasks. in addition to ci generation, such tasks also include couplet and classical poetry generation, where they have different concerns regarding to particular restrictions. specifically, couplet generation @xcite @xcite @xcite is a strictly conditioned text generation task where the generated text (subsequent clause) has to correspond to the input text (antecedent clause) in almost all aspects, such as rhyme, length, syntactic and semantic correspondence, etc. classical poetry generation @xcite @xcite @xcite @xcite normally focuses on unconditioned text generation with limited format constraints, where there are typically two poem types, i.e., five-character and seven-character quatrain. different from the two tasks, ci generation is more flexible than couplet but less than poem, in terms of using tune patterns for restriction. to the best of our knowledge, there are 871 different types of tune patterns with each having its own format requirements. in performing ci generation, studies are much less than that for couplet and poem generation, recent ones @xcite @xcite leverage deep learning based models and achieve outstanding performance, where most of them regard the task as a conventional sequence-to-sequence task and use autoregressive approaches. to further improve the task, there are studies applying enhanced modules such as attentions @xcite and pre-training language models @xcite . compared to previous studies based on deep neural networks, this work takes the advantage of the properties of ci (i.e., the format is determined once the tune pattern is given and different parts of ci should stick to a particular topic) and provides an alternative solution for ci generation through a nonautoregressive method, which allows our model to generate ci efficiently. particularly, the generation process guided by essential words and rl with carefully designed rewards further facilitate the explicit accommodation of the rigid constraints for ci, leading to better results in all evaluations. ## conclusion in this paper, we propose a non-autoregressive model named cigen for ci composition, which is further enhanced by an essential word guided generation process and rl. specifically, our model firstly generates the ess words that convey important meaning or topic of ci and then use these ess words to complement the generation of the entire ci. in addition, we design a set of rl rewards based on format, rhyme, and diversity (including uni-grams and bi-grams based measures) to enhance the model by further accommodating the constraints from the tune pattern, for the purpose of solving the problem that normal loss functions for conventional supervised/unsupervised methods cannot be applied to such constraints. experimental results and analyses on a song ci dataset confirm the validity of our proposed model, with its evaluation outperforms strong baselines and previous studies with respect to different evaluation metrics. moreover, owing to the non-autoregressive characteristic, the inference speed of our model also shows its great superiority over the autoregressive ones. therefore, the effectiveness and efficiency indicates that our model design has its potential to be implemented to similar text generation scenarios.