## 3 Methods
### Automatic Evaluation Model Based on BERT-CCPoem  
Automatically evaluating the quality of generated poems is also a very challenging issue. Human experts [12; 13; 14; 15; 16; 17; 18; 19] or the automatic metrics BLEU technique [2; 13; 14; 16; 17; 18; 20; 21] are the most prevalent approaches for assessing the quality of poems. The evaluation of the quality of a large number of poems by human experts is time-consuming and expensive. Moreover, different experts have their appreciation preferences for poems, so it inevitably  
Figure 2: The classical Chinese poetry generation model based on Transformer-XL (CCPTXL): (**a**) The classical Chinese poetry generation model and (**b**) Multi-Head Attention Layer.  
leads to a certain subjectivity. BLEU is an automated and inexpensive measure standard for machine translation that achieves a high correlation with human judgments of quality, but poetry generation is fundamentally different from machine translation. First, the source and target sentences in machine translation attempt to have the same meaning, but the poetry creation is different. The target sentence does not repeat the meaning of the source sentence, although it is based on the source sentence. Second, BLEU is a method based on word mechanical matching, which cannot compute the sentences relationship according to the semantic meaning of words. Thus, both tone-pattern-checker and text-fluency-checker are important in an automatic poetry evaluation model. The tone-pattern-checker can give a score after identifying whether a poem follows the tone and rhyming-rules of poetry. The algorithm of a tone-pattern-checker is shown in Algorithm 1, and consists of structure detection and tone rhyming detection. The structure detection is to determine whether a poem follows the format of the 5-character quatrain, 7-character quatrain, 5-character regulated verse, or 7-character regulated verse. The tone rhyming detection is to detect whether a poem obeys the alternative-rule, opposite-rule, alike-rule, and rhyming-rule, and finally gives a corresponding score.  
```
1:Input: one quatrain or regulated verse noted as X, tone_score = 0;
2:if the structure of X follows the rules of quatrain or regulated verse then
3:tone_score = 2
4:end if
5:if X follows the alternative-rule then
6:tone_score = tone_score + 2
7:end if
8:if X follows the opposite-rule,
9:tone_score = tone_score + 2;
10:end if
11:if X follows the alike-rule then
12:tone_score = tone_score + 2
13:endif
14:if X follows the rhyming-rule then
15:tone_score = tone_score + 2
16:end if
17:return tone_score.
``` ```  
**Algorithm 1** Tone-Pattern-Checker  
The text-fluency-checker is responsible for detecting whether a poem follows ancient Chinese grammatical norms, whether it reads fluently, whether the collocation of words is reasonable, and whether the sentences are coherent. Moreover, it gives corresponding quantitative scores to poems.  
By using the deep representations of ancient Chinese provided by the pre-trained BERT-CCPoem [33], we model poem fluency evaluation as a text classification problem, as shown in Figure 3. The BERT model is a transformer-based bidirectional mask language model that has achieved SOTA results on many natural language processing tasks, including GLUE, SQuADv1.1, SQuADv2.0, and others. BERT-CCPoem is a pre-trained BERT-based model for classical Chinese poetry by the Research Center for Natural Language Processing of Tsinghua University. BERT-CCPoem learns plentiful semantic and grammatical information on Chinese characters because it is trained on a large amount of the Chinese corpus. Therefore, using the semantic vectors provided by BERT-CCPoem, we add a classifier over BERT-CCPoem to fine-tune a text-fluency-checker.