C hip S ong: A Controllable Lyric Generation System for C hinese Popular Song (2022)

Abstract: In this work, we take a further step towards satisfying practical demands in Chinese lyric generation from musical short-video creators, in respect of the challenges on songs’ format constraints, creating specific lyrics from open-ended inspiration inputs, and language rhyme grace. One representative detail in these demands is to control lyric format at word level, that is, for Chinese songs, creators even expect fix-length words on certain positions in a lyric to match a special melody, while previous methods lack such ability. Although recent lyric generation community has made gratifying progress, most methods are not comprehensive enough to simultaneously meet these demands. As a result, we propose ChipSong, which is an assisted lyric generation system built based on a Transformer-based autoregressive language model architecture, and generates controlled lyric paragraphs fit for musical short-video display purpose, by designing 1) a novel Begin-Internal-End (BIE) word-granularity embedding sequence with its guided attention mechanism for word-level length format control, and an explicit symbol set for sentence-level length format control; 2) an open-ended trigger word mechanism to guide specific lyric contents generation; 3) a paradigm of reverse order training and shielding decoding for rhyme control. Extensive experiments show that our ChipSong generates fluent lyrics, with assuring the high consistency to pre-determined control conditions.

## introduction lyric generation is a recent emerging topic in intelligent music research community, which has attracted increasing attention and gained progress in the past few years @xcite @xcite @xcite @xcite . meanwhile, observing a large * corresponding author amount of music lovers, amateurs, and professional musicians are gathering on today's fast growing chinese short-video platforms (e.g., kwai, tiktok, wesee, etc.), where they create and post musical short-videos actively, with purpose to obtain more follows and likes from general population; we believe it is worth to customize a lyric generation system for their short-video display purpose. hence, in this paper, we aim to put more emphasis on assisting creators from practical shortvideo scenario with realistic demands. in oder to collect their real demands, a qualitative investigation with 85 potential users (ages: 18∼40 years; 42 female, 43 males; 19 full-time musicians, 66 part-time musicians) is conducted at the very first stage. here, we briefly release 4 representative demands as follow: 1) short lyric paragraphs are required to fit in short-video durations, mostly under 60 sec. @xcite ; 2) open-ended inspiration inputs are desired to guide specific content generation from various creators; 3) length format controlling at sentence and even word level is expected to strictly match melody length format for flexible creation intents, where a chinese word is generally composed of multiple characters (e.g., "爱" means love, "爱 好" means hobby, "爱 尔 兰" means ireland) and one character sounds one syllable; 4) rich rhyme patterns are needed for smooth song singing. although recent progress has been made on lyric generation, previous works are not comprehensive enough to simultaneously meet these customized demands; what's more, as far as we know, none of the existing work supports word-level length format control. taking the above challenges in mind, we develop chipsong, a lyric generation system, to assist musical short-video creators for chinese popular song creation. as shown in figure 1 , with chipsong, a creator is encouraged to input a group of openended words (which are referred to as trigger words in the following) to represent his/her inspiration, trigger word 思念;夜晚 (missing; night) rhyme {an; an; an; an} generated lyric of chipsong sentence-level length format {7; 7; 9; 9} word-level length format {2, 2, 1, 2; 1, 1, 1, 2, 2; 1, 1, 1, 1, 2, 1, 2; 2, 2, 3, 1, 1} example input template 每 个 失 眠 的 夜 晚 (wan) 谁 在 我 心 头 轻 叹 (tan) 是 爱 把 人 变 得 更 敏 感 (gan) 思 念 本 是 折 不 断 的 线 (xian) ( sleepless in every night in my heart a soft sigh it is love that makes people hide missing is a continuous line ) trigger word 女孩;迷人 (girl; charming) ## related work recent lyric generation works can be broadly categoried into three groups according to their cared artistic genres: 1) hip-pop generation, creating hippop lyrics with distinctive rhymes and rhythms constrains @xcite @xcite ; 2) poetry generation, creating some special text paradigms, such as shakespeare's sonnet @xcite , chinese classical poetry @xcite @xcite , and chinese couplet @xcite , etc; 3) popular song generation, creating full-text lyrics @xcite @xcite @xcite or polishing draft lyrics @xcite for popular songs. our chipsong is actually a lyric generation system within the third group, and especially for chinese popular songs. moreover, different from previous lyric generation works, which were mostly model-oriented for natural paragraphs generation and excluded explicit user profiles from practical application scenarios, chip-song customizes functions to generate lyrics for users from practical short-video scenario with re- lyric: 孤单的下雨天，而你已走远。 ( lonely rainy day, and you have gone far away. ) e0 天 e[o] e天 e1 雨 e[o] e雨 e2 [start] e[c] e[start] e3 [6] e[b-3] e[6] e4 天 e[i-3] e天 e5 下 e[e-3] e下 e6 的 e[b-1] e的 e7 单 e[b-2] e单 e8 孤 e[e-2] e孤 e9 [sep] e[s] e[sep] e10 [5] e[c] e[5] e11 远 e[b-2] e远 e12 走 e[e-2] e走 e13 已 e[b-1] e已 e14 你 e[b-1] e你 e15 而 e[b-1] e而 e16 [end] e[o] e[end] 雨 [start] ## conclusion in this work, we develop chipsong, a lyric generation system, to assist musical short-video creators for chinese popular song creation. chipsong fine-grainly adjusts lyric generation to meet the creator's needs in various scenarios via the proposed strategies of sentence-and word-level format control, trigger word control, and rhyme control. in the future, we would like to consider melody generation and other attributions control of lyrics.