C har P oet: A C hinese Classical Poetry Generation System Based on Token-free LLM (2024)

Abstract: Automatic Chinese classical poetry generation has attracted much research interest, but achieving effective control over format and content simultaneously remains challenging. Traditional systems usually accept keywords as user inputs, resulting in limited control over content. Large language models (LLMs) improve content control by allowing unrestricted user instructions, but the token-by-token generation process frequently makes format errors. Motivated by this, we propose CharPoet, a Chinese classical poetry generation system based on token-free LLM, which provides effective control over both format and content. Our token-free architecture generates in a character-by-character manner, enabling precise control over the number of characters. Pruned from existing token-based LLMs, CharPoet inherits their pretrained capabilities and can generate poetry following instructions like “Write me a poem for my mother’s birthday.” CharPoet achieves format accuracy above 0.96, outperforming Jiuge-GPT-2 (0.91) and GPT-4 (0.38). In terms of content quality, CharPoet surpasses traditional systems including Jiuge, and is comparable to other LLMs. Our system is open source and available at https://modelscope.cn/models/CharPoet/CharPoet . A video demonstration of CharPoet is available at https://youtu.be/voZ25qEp3Dc .

## introduction chinese classical poetry, one of the most valuable heritages of human culture, conveys rich connotations through its concise and exquisite form. chinese classical poetry can be classified into two primary categories: shi and ci, both of which have strict format requirements @xcite . for example, wuyanjueju, the simplest form of shi, requires four lines with each line containing exactly five chinese characters. ci is more complex: there are nearly one thousand forms in total, each with different requirements for the number of lines and characters. automatic generation of chinese classical poetry has attracted much research interest. however, achieving effective control over both format and content simultaneously remains a challenge. traditional systems in this field usually take keywords as user inputs @xcite @xcite @xcite @xcite . however, it is often insufficient for users to fully describe the theme or emotion they expect with just one or several keywords. this inability to process complex inputs has reduced the diversity and quality of the generated poetry. in contrast, large language models (llms) can accept unrestricted user prompts and allow more control over the content. llms are capable of generating diversified texts following complex user instructions @xcite @xcite . nevertheless, token-based llms face challenges in strictly adhering to the expected format of poetry, occasionally producing lines with an excess or insufficient number of characters. an example of a gpt-4-generated poem is given in figure 1 . in this example, gpt-4 is asked to write a poem in the rumengling form, with the keyword cheerful. the generated poem performs well in terms of content, but it clearly violates the format requirements. the redundant characters are marked in red with a strikethrough. we argue that the problem is partly due to the token-based nature of llms. standard token-based llm systems split text into word pieces before feeding them into the model. these text pieces are known as tokens, and they usually contain more than one character @xcite . the system must generate text in a token-by-token manner. under such a setting, if a model needs to control the number of characters precisely, it must know exactly how many characters are contained in each token. we have conducted a simple test that shows llms clearly lack such knowledge. the results are provided in appendix a. motivated by this, we propose charpoet, a chinese classical poetry generation system based on a token-free llm, which achieves effective control over both format and content simultaneously. "token-free" here means that our model operates only on characters or bytes, in contrast to regular tokens. as shown in figure 2 , our system generates poems in a character-by-character manner. with the token-free architecture, our system can precisely control the number of characters. instead of being trained from scratch, our token-free llm is pruned from existing token-based models. we remove long tokens from the tokenizer and the language model head, keeping only character-level and byte-level tokens, and then finetune on a poetry dataset. through this pruning process, our system inherits capabilities from existing tokenbased llms, and can generate poetry following complex instructions such as "write me a poem for my mother's birthday." without any post-processing, our token-free system achieves a format accuracy of 0.96, outperforming jiuge-gpt-2 (0.91) and gpt-4 (0.38). in addition, our system performs comparably to existing llms in terms of the content quality. ## related work traditional systems in this field @xcite @xcite @xcite have demonstrated that rnns and lstms can generate high-quality poetry. however, these systems usually accept keywords as user inputs, resulting in poor control over content. moreover, they often have complex architectures or special modules designed to handle the strict format and content constraints inherent in poetry. for example, @xcite imposes a working memory mechanism; @xcite implements a postprocess module to filter poems with unexpected format. large language models (llms) (openai, 2022, 2023; the qwen team, 2023) have demonstrated the power of the transformer architecture @xcite when trained with a large corpus. llms are capable of generating high-quality and diversified poetry following unrestricted prompt. however, they suffer from problems with format accuracy due to their token-based nature. more in line with our research, @xcite ; @xcite build poetry generation systems based on token-free language models. however, those systems are trained from scratch so they do not inherit the great power from pretrained llms. they still accept keywords as user inputs and cannot understand complex instructions. figure 3 : prune a token-based model into a token-free one. (a) for input, long tokens will be removed from the vocabulary. text would only be tokenized into character-level or byte-level tokens; the embeddings of long tokens will never be accessed. (b) transformer structure is left unchanged. (c) for output, the logits of long tokens will be set to a large negative number and the probabilities of long tokens will be zero. the language model head would never produce long tokens. ## demonstration the user interface of our poetry generation system is shown in figure 4 . in contrast to previous systems where users need to summarize the theme of the poetry they want in one or several keywords, our system allows users to describe desired content with natural language in the prompt box. after that, the user selects a poetry form and clicks the "submit" button. a few seconds later, the system returns a poem following the user's instruction. our system is fully open source, available at https://modelscope.cn/models/charpoet/ charpoet . we have included a jupyter notebook in the project. using this notebook, anyone can launch the application and try our system. we also provide some example poems in appendix c. ## conclusion in this paper, we address the problem of achieving effective control over both format and content in the field of automatic chinese classical poetry generation. we propose a token-free system char-poet, which generates in a character-by-character manner, enabling precise control over the number of characters. moreover, charpoet allows for human instructions in natural language, in contrast to traditional models that only accept keywords. charpoet achieves format accuracy above 0.96 without any postprocessing, higher than jiuge-gpt-2 (0.91) and gpt-4 (0.38). our ablation study comparing charpoet with its token-based equivalent shows that the token-free architecture brings a 3% gain in format accuracy. in addition, our system's performance in content quality surpasses traditional systems, and is comparable to existing llms.