Q iu N iu: A C hinese Lyrics Generation System with Passage-Level Input (2022)

Abstract: Lyrics generation has been a very popular application of natural language generation. Previous works mainly focused on generating lyrics based on a couple of attributes or keywords, rendering very limited control over the content of the lyrics. In this paper, we demonstrate the QiuNiu, a Chinese lyrics generation system which is conditioned on passage-level text rather than a few attributes or keywords. By using the passage-level text as input, the content of generated lyrics is expected to reflect the nuances of users’ needs. The QiuNiu system supports various forms of passage-level input, such as short stories, essays, poetry. The training of it is conducted under the framework of unsupervised machine translation, due to the lack of aligned passage-level text-to-lyrics corpus. We initialize the parameters of QiuNiu with a custom pretrained Chinese GPT-2 model and adopt a two-step process to finetune the model for better alignment between passage-level text and lyrics. Additionally, a postprocess module is used to filter and rerank the generated lyrics to select the ones of highest quality. The demo video of the system is available at https://youtu.be/OCQNzahqWgM .

## introduction ai creation is an important application domain of natural language generation (nlg), including story generation @xcite , poetry writing @xcite @xcite , lyrics generation @xcite @xcite , etc,. particularly, lyrics generation has always been a popular task of nlg since its intrusiveness and easy data availability. previous works of lyrics generation @xcite @xcite @xcite on specified keywords (e.g., flower) or certain attributes such as the lyrics' text style (e.g., hip-hop) and expected theme described by the lyrics (e.g., love). however, these input only provide very limited control over the content of generated lyrics. sometimes the generated lyrics may deviate far from the user's needs. to improve the usability of ai as a creation tool, we need to improve the controllability of the generated content. we argue that adopting free form text as the input is an approach to having precise control over the content of generated lyrics. as seen in figure 1 , an author usually conceives a passage (shown in the left text box) in his/her mind that expresses his/her inner feelings and thoughts, and then uses a wealth of writing skills and rhetorical techniques to create the final work (shown in the right text box). in this paper, we demonstrate qiuniu (the eldest son of the dragon in ancient chinese mythology, who loves music), a chinese lyrics generation system conditioned on free form passage-level text. the qiuniu system can receive various forms of passage-level user input, which may be in different passage-level user input short story essay classical poetry modern poetry generation model 十七岁那年 at the age of seventeen 男孩花了半年的积蓄 the boy spent half a year of his savings 陪女孩去听演唱会 accompany the girl to the concert …… postprocess 𝑬𝒏𝒄 𝒔 for passagelevel text 𝑫𝒆𝒄 𝒕 for lyrics 她来听我的演唱会 she came to my concert 在十七岁的初恋第一次约会 first date at the age of seventeen 男孩为了她彻夜排队 boy queued all night for her 半年的积蓄买了门票一对 half a year of savings to buy a pair of tickets …… relevance reranking anti-spam lyrics scoring genres (e.g., short stories, essays, poetry), and eras (e.g., classical poetry, modern poetry). it is basically a text style transfer problem, which greatly suffers from the lack of aligned corpus. to construct the training data, we collected a passage-level corpus d s from multiple sources and 300k different styles of lyrics corpus d t . note that it is intractable to train a sequence-to-sequence (seq2seq) model from passage-level text to lyrics directly because the d s and d t are not aligned. to address the issue, the qiuniu system adopts the framework of unsupervised machine translation (umt) @xcite . specifically, the framework consists of an encoder enc s and a decoder dec s for the input side, an encoder enc t and a decoder dec t for lyrics side. the encoder enc s (or enc t ) encodes the passage-level input text (or lyrics) into a hidden representation and the decoder dec s (or dec t ) decodes it into lyrics (or passage-level text). the objective of the model training is to align the passage-level text and lyrics in the latent representation space. to train the model, we first initialize the parameters with a custom chinese gpt-2 @xcite model, which is pretrained on around 30g chinese books corpus collected online. then we adopt a two-step process to finetune the model by jointly optimizing self-reconstruction loss, crossreconstruction loss and alignment loss. after the training is finished, enc s encodes the passage-level input and dec t generates the candidate lyrics. finally, a postprocess module is used to filter and rerank the generated lyrics to select the ones of highest quality. human evaluation indicates the effectiveness of the framework. the contributions of the qiuniu system are summarized as follows: ## architecture the architecture of qiuniu system is shown in figure 2. it mainly consists of three modules: passagelevel user input, generation model and postprocess. each module is described in detail below. ## conclusion in this paper, we demonstrate qiuniu, a chinese lyrics generation system conditioned on passagelevel input. we support various forms of passagelevel input, covering different genres and eras. the qiuniu system adopts the framework of unsupervised machine translation due to the lack of aligned corpus from passage-level text to lyrics. besides, the model of qiuniu is initialized with the pretrained chinese gpt-2 parameters and finetuned in a two-step process to improve the alignment between the passage-level text and lyrics. finally, a postprocess module is used to filter and rerank the generated lyrics to select the high-quality ones.