CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLM (2024)

Abstract: Automatic Chinese classical poetry generation has attracted much research interest, but achieving effective control over format and content simultaneously remains challenging. Traditional systems usually accept keywords as user inputs, resulting in limited control over content. Large language models (LLMs) improve content control by allowing unrestricted user instructions, but the token-by-token generation process frequently makes format errors. Motivated by this, we propose CharPoet, a Chinese classical poetry generation system based on token-free LLM, which provides effective control over both format and content. Our token-free architecture generates in a characterby-character manner, enabling precise control over the number of characters. Pruned from existing token-based LLMs, CharPoet inherits their pretrained capabilities and can generate poetry following instructions like “Write me a poem for my mother’s birthday.” CharPoet achieves format accuracy above 0.96, outperforming Jiuge-GPT-2 (0.91) and GPT-4 (0.38). In terms of content quality, CharPoet surpasses traditional systems including Jiuge, and is comparable to other LLMs. Our system is open source and available at https://modelscope. cn/models/CharPoet/CharPoet. A video demonstration of CharPoet is available at https://youtu.be/voZ25qEp3Dc.

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 315–325 August 11-16, 2024 ©2024 Association for Computational Linguistics
Automatic Chinese classical poetry generation has attracted much research interest, but achieving effective control over format and content simultaneously remains challenging. Traditional systems usually accept keywords as user inputs, resulting in limited control over content. Large language models (LLMs) improve content control by allowing unrestricted user instructions, but the token-by-token generation process frequently makes format errors. Motivated by this, we propose CharPoet, a Chinese classical poetry generation system based on token-free LLM, which provides effective control over both format and content. Our token-free architecture generates in a characterby-character manner, enabling precise control over the number of characters. Pruned from existing token-based LLMs, CharPoet inherits their pretrained capabilities and can generate poetry following instructions like “Write me a poem for my mother’s birthday.” CharPoet achieves format accuracy above 0.96, outperforming Jiuge-GPT-2 (0.91) and GPT-4 (0.38). In terms of content quality, CharPoet surpasses traditional systems including Jiuge, and is comparable to other LLMs. Our system is open source and available at https://modelscope. cn/models/CharPoet/CharPoet. A video demonstration of CharPoet is available at https://youtu.be/voZ25qEp3Dc.

heading: 1 Introduction
text: Chinese classical poetry, one of the most valuable heritages of human culture, conveys rich connotations through its concise and exquisite form. Chinese classical poetry can be classified into two primary categories: SHI and CI, both of which have strict format requirements (Hu and Sun, 2020). For example, Wuyanjueju, the simplest form of SHI, requires four lines with each line containing exactly
∗Equal contribution. †Corresponding authors.
five Chinese characters. CI is more complex: there are nearly one thousand forms in total, each with different requirements for the number of lines and characters.
Automatic generation of Chinese classical poetry has attracted much research interest. However, achieving effective control over both format and content simultaneously remains a challenge.
Traditional systems in this field usually take keywords as user inputs (Guo et al., 2019; Hu and Sun, 2020; Wang et al., 2016; Yan, 2016; Yi et al., 2017, 2018; Zhang and Lapata, 2014; Zhang et al., 2017). However, it is often insufficient for users to fully describe the theme or emotion they expect with just one or several keywords. This inability to process complex inputs has reduced the diversity and quality of the generated poetry. In contrast, Large Language Models (LLMs) can accept unrestricted user prompts and allow more control over the content. LLMs are capable of generating diversified texts following complex user instructions (OpenAI, 2022, 2023; the Qwen team, 2023). Nevertheless,
315
token-based LLMs face challenges in strictly adhering to the expected format of poetry, occasionally producing lines with an excess or insufficient number of characters.
An example of a GPT-4-generated poem is given in Figure 1. In this example, GPT-4 is asked to write a poem in the Rumengling form, with the keyword cheerful. The generated poem performs well in terms of content, but it clearly violates the format requirements. The redundant characters are marked in red with a strikethrough.
We argue that the problem is partly due to the token-based nature of LLMs. Standard token-based LLM systems split text into word pieces before feeding them into the model. These text pieces are known as tokens, and they usually contain more than one character (Sennrich et al., 2016; Schuster and Nakajima, 2012). The system must generate text in a token-by-token manner. Under such a setting, if a model needs to control the number of characters precisely, it must know exactly how many characters are contained in each token. We have conducted a simple test that shows LLMs clearly lack such knowledge. The results are provided in Appendix A.
Motivated by this, we propose CharPoet, a Chinese classical poetry generation system based on a token-free LLM, which achieves effective control over both format and content simultaneously. “Token-free” here means that our model operates only on characters or bytes, in contrast to regular tokens. As shown in Figure 2, our system generates poems in a character-by-character manner. With the token-free architecture, our system can precisely control the number of characters. Instead of being trained from scratch, our token-free LLM is pruned from existing token-based models. We remove long tokens from the tokenizer and the language model head, keeping only character-level and byte-level tokens, and then finetune on a poetry dataset. Through this pruning process, our system inherits capabilities from existing tokenbased LLMs, and can generate poetry following complex instructions such as “Write me a poem for my mother’s birthday.”
Without any post-processing, our token-free system achieves a format accuracy of 0.96, outperforming Jiuge-GPT-2 (0.91) and GPT-4 (0.38). In addition, our system performs comparably to existing LLMs in terms of the content quality.

heading: 2 Related work
text: Traditional systems in this field (Zhang and Lapata, 2014; Wang et al., 2016; Yan, 2016; Yi et al., 2017, 2018; Guo et al., 2019) have demonstrated that RNNs and LSTMs can generate high-quality poetry. However, these systems usually accept keywords as user inputs, resulting in poor control over content. Moreover, they often have complex architectures or special modules designed to handle the strict format and content constraints inherent in poetry. For example, Yi et al. (2018) imposes a working memory mechanism; Guo et al. (2019) implements a postprocess module to filter poems with unexpected format.
Large Language Models (LLMs) (OpenAI, 2022, 2023; the Qwen team, 2023) have demonstrated the power of the Transformer architecture (Vaswani et al., 2017) when trained with a large corpus. LLMs are capable of generating high-quality and diversified poetry following unrestricted prompt. However, they suffer from problems with format accuracy due to their token-based nature.
More in line with our research, Hu and Sun (2020); Belouadi and Eger (2023) build poetry generation systems based on token-free language models. However, those systems are trained from scratch so they do not inherit the great power from pretrained LLMs. They still accept keywords as user inputs and cannot understand complex instructions.

heading: 3 Architecture

heading: 3.1 Pruning
text: The core of our system is a token-free LLM. Instead of being trained from scratch as in previous work (Belouadi and Eger, 2023), our token-free LLM is pruned from an existing token-based LLM to inherit the pretrained knowledge and capabilities. Our token-free model accepts unrestricted prompts as input and returns poems that excel in both format accuracy and content quality.
We have designed a procedure that can prune any typical token-based LLM into a token-free model. A typical LLM, such as Llama (the Llama team, 2023) and Qwen (the Qwen team, 2023), contains three components, the Input (the tokenizer and the input embeddings), the Transformer (Vaswani et al., 2017) and the Output (the language model head). Our pruning procedure modifies the Input component and the Output component, and leaves the Transformer component unchanged. The procedure is described below and illustrated in Figure 3.
(a) Input Pruning. We prune the tokenizer’s vocabulary by removing all long tokens, leaving only character-level or byte-level fragments. Long tokens refer to two types: tokens with more than one Chinese character and tokens consisting of a single Chinese character combined with non-Chinese characters. Once these tokens are removed, the
tokenizer will only produce character-level or bytelevel fragments. Subsequently, the input embeddings for these removed tokens will never be accessed or updated.
We retain non-Chinese tokens as they are. This approach ensures that the keywords commonly used in LLM chat settings like “user” and “assistant”, remain intact to preserve the standard tokenization of chat templates.
(b) Transformer kept unchanged. The structure of the Transformer is left unchanged, while the parameters will still be updated during finetuning.
(c) Output Pruning. For outputs, we set the probabilities of all long tokens to zero. This is achieved by incorporating an indicator function into the original softmax transformation:
Prob(ti) = (1− 1L(i)) exp(logiti)∑ j(1− 1L(j)) exp(logitj)
where logiti denotes the neural network’s output value of the ith token prior to the softmax transformation, and the indicator function determines if the ith token is a member of the long token set L.
1L(x) = { 1 if x ∈ L 0 if x /∈ L
In practice, we implement this by adding a large negative number to the logits of long tokens, instead of modifying the softmax function directly.
With the above procedure, any typical LLM could be pruned to a token-free model. In contrast to typical token-based LLMs, the pruned token-free model outputs text in a character-by-character manner and is expected to perform better on charactersensitive tasks such as poetry generation.
In this paper, we use Qwen-7B-Chat (the Qwen team, 2023) as the base model. An interesting observation is that even without further finetuning, the pruned token-free LLM is already capable of answering simple questions. We provide some examples in Appendix B. Nevertheless, we suggest further finetuning on the target dataset for better performance.

heading: 3.2 Training
text: Training involves two stages: general-purpose training and poetry-field training.

heading: 3.2.1 General Purpose Training
text: We need general-purpose training because our model is directly pruned from an existing tokenbased LLM and not familiar with natural language presented at character level. Here we use BELLE dataset (Ji et al., 2023), which is a high-quality general-purpose instruction-following dataset.

heading: 3.2.2 Poetry-field Training
text: In the second stage, we train with our in-house poetry dataset. The dataset contains 20000 human written poems, and each poem is created based on an input prompt. The prompts cover a broad range of topics, including specific scenes, emotions, and both concrete and abstract themes.
To improve format accuracy, we provide the model with a masked version of the expected poem as a format hint. In this masked poem, all
Chinese characters are replaced with a mask sign [M] while punctuation and line breaks are kept as is. An example of a masked poem in the form Rumengling is provided in below.
[M][M][M][M][M][M]， [M][M][M][M][M][M]。 [M][M][M][M][M]， [M][M][M][M][M][M]。 [M][M]， [M][M]， [M][M][M][M][M][M]。
The masked poem is provided together with the original user prompt. The final prompt-response format is designed as follows, where [SOP] denotes start of piece, [EOP] denotes end of piece, {ORIGINAL USER PROMPT} denotes the original prompt from the user, {MASKED POEM} is the format hint, and {POEM} denotes the generated poem.
[SOP]user Fill in all the masks [M]. {ORIGINAL USER PROMPT} Output: {MASKED POEM} [EOP] [SOP]assistant {POEM} [EOP]
In this way, the poem-generating task is transformed into a mask-filling task. With the token-free architecture, our model fills in all the masks in a
character-by-character manner. The mask-filling design ensures that the model can strictly follow the format constraints of the requested poetry type.

heading: 4 Demonstration
text: The user interface of our poetry generation system is shown in Figure 4. In contrast to previous systems where users need to summarize the theme of the poetry they want in one or several keywords, our system allows users to describe desired content with natural language in the prompt box. After that, the user selects a poetry form and clicks the “Submit” button. A few seconds later, the system returns a poem following the user’s instruction.
Our system is fully open source, available at https://modelscope.cn/models/CharPoet/ CharPoet. We have included a Jupyter notebook in the project. Using this notebook, anyone can launch the application and try our system. We also provide some example poems in Appendix C.

heading: 5 Evaluation

heading: 5.1 Test settings
text: We evaluate performance on two aspects: format accuracy and content quality.
For comparability with previous studies, we assess performance on four types of SHI and six types
of CI, as in Hu and Sun (2020) exactly. To better study the relationship between format accuracy and the length of poetry, we have additionally included three popular types of form in the evaluation set, which are Busanzi, Pusaman and Shuidiaogetou.
We conduct tests under two user input settings: the first is the conventional keyword setting, where the user input consists of a single keyword; the second is the instruction setting, where the user input is a natural language instruction, such as “Write me a poem for my mother’s birthday.”. In both settings, one specific format is selected as the expected format.
We conduct 100 tests for each type of form and each setting. In the keyword setting, we use a collection of 100 frequently used Chinese idioms sourced from the internet. Chinese idioms convey rich meanings in simple expressions, and are thus more challenging than regular words. In the instruction setting, we ask GPT-4 to generate 100 prompts as user inputs. We have double-checked the GPTgenerated prompts; they cover a broad range of topics, including specific scenes, emotions, and both concrete and abstract themes.
We do not use human-written prompts because human researchers could potentially manipulate the prompt set to alter research conclusions. For example, human researchers may remove the prompts
where the model performs poorly, making the evaluation scores appear better. By directly using GPTgenerated prompts without any modifications, we can effectively avoid biases caused by such manipulation issues.

heading: 5.2 Models for Comparison
text: We compare our system CharPoet with two categories of public available systems. One category is general-purpose LLMs, with GPT-4 (OpenAI, 2023) being the top performer. The other category is systems exclusively designed for automatic poetry generation, with Jiuge (Guo et al., 2019) being the most representative.
GPT-4. To exploit GPT-4’s potential in format accuracy, we have carefully designed the prompt. We find that GPT performs better if provided with an example poem of the required form. The prompt template is provided in Appendix D.
Jiuge. Jiuge (Guo et al., 2019) is a comprehensive system with a postprocessing module to ensure format accuracy; therefore, when evaluating format accuracy, we compare instead with Jiuge-GPT-2 (Hu and Sun, 2020), the most recent work in the Jiuge series, which is more comparable since it is transformer-based and end-to-end.
Ablation study. To verify the effectiveness of our token-free architecture, we also conducted an ablation study, where we compared our system to its token-based equivalent. The token-based equivalent is identical to CharPoet in every aspect including model size, prompt design and training dataset, except that it is built on the original tokenbased Qwen-Chat (the Qwen team, 2023) instead of our pruned token-free version. The token-based equivalent is marked as Qwen (Finetuned) in corresponding tables and figures.

heading: 5.3 Evaluation on Format Accuracy
text: Format accuracy results are shown in Table 1. A poem is counted as accurate only if the number of characters for every line is correct (perfect match). The figures for Jiuge-GPT-2 are directly collected from the original paper, while the figures for other models are obtained from our testing procedure. The figures for Jiuge-GPT-2 under the instruction setting are not available since Jiuge-GPT-2 does not support user instructions. For comparability, the average accuracy is calculated based on the overlapping 10 types of poetry, rather than all 13 types.
CharPoet performs better in format accuracy than all competing models, achieving an overall accuracy above 0.96 under both settings. Our ablation study comparing CharPoet with its tokenbased equivalent Qwen (Finetuned) confirms that the token-free architecture is effective, bringing a 3% gain in format accuracy.
Consistent with Hu and Sun (2020), our results show that SHI is simple and all models listed here achieve decent accuracy. As for CI, which is more complex and challenging, our system beats previous systems by a large margin. For example, in terms of Qinyuanchun, the longest type of poem in our test set, our system achieves 0.84 accuracy, compared to 0.55 of Jiuge-GPT-2 and nearly zero of GPT-4. Regression analysis also indicates that CharPoet is less sensitive to poem length (See Appendix E for details).

heading: 5.4 Evaluation on Content Quality
text: Following Yi et al. (2018), we evaluate content quality with five criteria; each criterion needs to be scored on a 5-point scale:
Fluency. Does the poem obey the grammatical, structural and phonological rules?
Meaning. Does the poem convey some certain messages?
Coherence. Is the poem as a whole coherent in meaning and theme?
Relevance. Does the poem express user topics well?
Aesthetics. Does the poem have some poetic and artistic beauties?
We first ask GPT-4 to conduct the scoring process. Though we have seen in the previous section that GPT-4 performs poorly in a poetry format, it remains top-notch in terms of content quality, making it a qualified evaluator for content assessment. The GPT-4 results under the two settings are shown in Figure 5 and Figure 6. The performance of CharPoet is basically the same as that of Qwen (Finetuned) and not far from GPT-4, while it significantly surpasses Jiuge, especially in terms of Relevance. The gain in content relevance indicates that pretrained LLMs can provide significantly better control over content compared to traditional models.
To ensure the reliability of GPT’s evaluation, we have also engaged human evaluators for doublechecking. We ask human labelers to score a subset of the evaluation set independently (without refer-
ring to GPT-4). The human results under the two settings are shown in Figure 7 and Figure 8. The results are in general consistent with GPT-4. We have also calculated the correlations between human and GPT-4 judgments using Pearson, Spearman, and Kendall-Tau. All correlations are greater than 0.5 with p-values less than 0.01, indicating that GPT-4 is a qualified evaluator in our settings.

heading: 6 Conclusion
text: In this paper, we address the problem of achieving effective control over both format and content in the field of automatic Chinese classical poetry generation. We propose a token-free system CharPoet, which generates in a character-by-character manner, enabling precise control over the number of characters. Moreover, CharPoet allows for human instructions in natural language, in contrast to traditional models that only accept keywords.
CharPoet achieves format accuracy above 0.96 without any postprocessing, higher than Jiuge-GPT2 (0.91) and GPT-4 (0.38). Our ablation study comparing CharPoet with its token-based equivalent shows that the token-free architecture brings a 3% gain in format accuracy. In addition, our system’s performance in content quality surpasses traditional systems, and is comparable to existing LLMs.

heading: 7 Limitations
text: Rhyme. In this paper, we propose the tokenfree method to enhance format accuracy. As a side effect, the token-free method may also enhance rhyme, which is also a highly character-sensitive task. Rhyme is an important aspect in Chinese classical poetry and deserves further study. Though rhyme is somehow covered in our evaluation process as part of the phonological rule in the fluency criterion, it deserves direct research with specially designed criteria and detailed indicators, and this is left to future work.
Other character-sensitive tasks & general ablilities. We have proposed a simple method to convert a pretrained token-based language model to a token-free one. It may be interesting to further investigate how the converted model performs in other character-sensitive tasks, such as named entity recognition and spelling correction. It may also be interesting to investigate how much general knowledge and abilities are retained during the conversion. These topics are also left to future work.

heading: A Probing into LLM’s knowledge in token-character relationship
text: For a token-based LLM, if it needs to control the number of characters precisely, it must know exactly how many characters are contained in each token. We have conducted a simple test, which shows that LLMs clearly lack such knowledge.
A.1 Method
Following Itzhak and Levy (2022), we use a probing procedure called “spelling bee” to investigate how much a LLM knows about the tokencharacter relationship of its vocabulary. Specifically, we probe whether the model has the knowledge that the token “大模型” contains three characters “大”, “模” and “型”.
The models we investigate here are the Qwen-series (the Qwen team, 2023), including Qwen-1.7B-chat, Qwen-7B-chat and Qwen-14Bchat. The Qwen series is one of the earliest open-source LLMs with a strong ability in Chinese and is influential in the Chinese community. In the context of large language models, the probing procedure could be formulated as an instruction following task, designed as follows.
Prompt List all the characters in the following token: 
大模型
Response 大
模
型
Here the special symbol 
 is used to ensure that both the long token in the prompt and the single characters in the response are tokenized as they are. We randomly selected 1000 tokens from the vocabulary to serve as a test set, and the remaining tokens are used as training examples.
Our procedure is not exactly the same as previous work (Kaushal and Mahowald, 2022; Itzhak and Levy, 2022). The main differences are
1. Our experiment probes all language model parameters, while previous work (Kaushal and Mahowald, 2022; Itzhak and Levy, 2022) probes only the vocabulary embedding, which ignores the knowledge contained in later layers, and thus would underestimate the real knowledge level.
2. In the context of large language models, we are able to conduct probing experiments relatively easily with the model itself through supervised finetuning (Ouyang et al., 2022), while previous work probes with a separate model, which may be difficult to train and would also underestimate the real knowledge level.
To sum up, we believe that our procedure can better estimate how LLM knows about the tokencharacter relationship relationship.
A.2 Results The results of the spelling bee probing procedure are summarized with the overall failure rate, which is defined as the number of times that the LLM fails to correctly output the character sequence of the required token, divided by the size of the test set (which is 1000 in our experiment). We also pay attention to another version of the failure rate, where we count an output as a failure only if the number of characters in the output does not equal the length of the original token. This metric is more relevant since it directly influences the control of character numbers in poem generation.
We have several observations:
1. Failure rates of token-based models are nontrivial. Failure rate near 1% seems small, but the influence is significant since mistakes would accumulate as the poem gets long.
2. The failure rate of token-based models seems not to vanish totally as model size increases.
3. The token-free model has reduced the failure rate by more than 90%.
B Token-free LLM without further finetuning
Without further finetuning the pruned token-free LLM is already capable of answering simple questions. Examples are shown below.
User: 简要介绍一下日心说
Agent: 日心说，也称日心地动说，是16th century末期由哥白尼提出的天体 学说，它将日、月、水星、金 星、木星、土星、天王星、海王 星和冥王星等9大天体，及小天体 如彗星、小星体等，都绕着以日 为圆心的椭圆轨转。
User: Briefly introduce the heliocentric theory.
Agent: The heliocentric theory is a model of the solar system that places the Sun at the center, with the planets orbiting around it in椭ical paths. This theory was proposed by Nicolaus Copernicus in the 16th century, but it was not widely accepted until the work of later astronomers such as Johannes Kepler and Galileo Galilei. The heliocentric model is now the most widely accepted model of the solar system, and it is supported by a vast amount of observational evidence.
We can see that the main meaning is correct, although there are some typos and a mix of Chinese and English.

heading: C Example Poems Generated by CharPoet
text: In the first example, our system is asked to write a poem with the keyword cheerful and the required form is Rumengling. This is exactly how we test GPT-4 in the Introduction section. We can see that poem generated by our system precisely match the required format of Rumengling, in contrast to GPT-4.
笑口频开深院，更说秋风天气。 With a frequent smile in the deep
courtyard, I praise the autumn weather.
心事向人知，却好兴高采烈。 I share thoughts with others, in high
spirits and delight. 休觅，休觅，酒到不知醒地。
Seek no more, seek no more. Drink till you do not know where you are.
More importantly, CharPoet allows for unrestricted instructions. In the following example, the prompt is “Write me a poem for my mother’s birthday.” and the required form is Rumengling. The output follows the expected format and content.
生日恰逢今日，母爱万金难拟。 Today is your birthday. A mother’s love is priceless. 恩重更情浓，岁岁同歌同醉。 Your kindness is profound. We sing and celebrate together, in each passing year. 同醉，同醉，寿星高上天际。 Drink together, drink together. You are the star today.

heading: D Test the Performance of GPT-4 on Format Accuracy
text: To exploit GPT-4’s potential in format accuracy, we have carefully designed the prompt. We find that GPT performs better if we provide it with an example poem of the required form. Our prompt is designed as follows.
Prompt 请写一首如梦令，主题或要求 为“兴高采烈”。请严格按照如梦令 对每一句话的字数要求，下面给 出一个例子： 常记溪亭日暮，沉醉不知归路。 兴尽晚回舟，误入藕花深处。 争渡，争渡，惊起一滩鸥鹭。
Prompt(translated into English) Please write a poem in the form “Rumengling”. The theme or instruction is “cheerful”. Please strictly follow the number of character requirements for each line. Here is an example:
I often recall the sun setting at the riverside pavilion, lost in the intoxication and unaware of the way back.
Later on when my excitement wanes, I return
on the boat, only to find myself unwittingly entering
a lotus pond. Struggling to cross,
struggling to cross, with seagulls and herons startled
by me and flew away.

heading: E Relationship between Format Accuracy and Poem Length.
text: We performed a regression analysis to investigate how format accuracy changes with poem length. Results show that in general the format accuracy decreases as the poem length increases. Results also show that CharPoet is less sensitive to poem length compared to competing models.

heading: Future Work
text: Though rhyme is somehow covered in our evaluation process as part of the phonological rule in the fluency criterion, it deserves direct research with specially designed criteria and detailed indicators, and this is left to future work. Other character-sensitive tasks & general ablilities. We have proposed a simple method to convert a pretrained token-based language model to a token-free one. It may be interesting to further investigate how the converted model performs in other character-sensitive tasks, such as named entity recognition and spelling correction. It may also be interesting to investigate how much general knowledge and abilities are retained during the conversion. These topics are also left to future work