## 4 Experiment and Results
### Evaluation for Generated Poems  
In order to fine-tune a text-fluency-checker, we first needed to construct positive and negative data sets. It was easy to build the positive data set by directly extracting some poems from the classical Chinese poems' dataset, whereas it was hard to create a negative data set of numerous poems written by humans. Therefore, we introduced noise into the manual poetry, which caused the original correct word order to be disrupted, resulting in a certain degree of grammatical errors, thereby automatically generating a large number of \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layers** & **Output Size** & **Trainable Parameters** \\ \hline Embedding layer & (100, 64, 410) & 4,137,310 \\ Multi-head layer 1  & (100, 64, 410) & 10,758,400 \\ Feed Forward layer 1  & (100, 64, 410) & 27,552,000 \\ Linear\_softmax layer & (100, 64, 10,091) & 4,137,310 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The model structure and parameters. Figure 3: The text-fluency-checker fine-tuned on BERT-CCPoem. negative poems. In addition, we were able to control the degree of grammatical errors in the negative data set according to the amount of noise introduced. From the classical Chinese poems dataset, we randomly extracted 12,000 poems that consisted of 6000 5-character and 6000 7-character regulated verse poems. The selected dataset consisted of 3 levels. Level 0 was a completely correct human-made poetry dataset without noise; Level 1 introduced 30% noise; and Level 2 introduced 60% noise on the original poetry dataset. To fine-tune the BERT-CCPoem model, we use the training dataset of 3 levels, which amounts to 36,000 poems. We also generated the validation dataset and testing dataset, which amounts to 7200 and 3600 poems, correspondingly. The text-fluency-checker gives a total score of 10, and each poem can get 10, 7, or 4 points corresponding to Level 0, Level 1, and Level 2, respectively. In the tone-checker, each poem also receives a total score of 10, with 2 points for the structure of quatrain or regulated verse, 2 points for the alternative-rule, 2 points for the opposite-rule, 2 points for the alike-rule, and 2 points for the rhyming-rule. We generated two 1000-poems-datasets by our CCPTXL model and a classical Chinese poetry model based on sequence to sequence with attention (CCPSA), respectively. Moreover, we also randomly extracted 1000 human-made poems from the classical Chinese Poetry Dataset (CCPD). Each of the three evaluation datasets consisted of 500 5-character and 500 7-character regulated verse. Besides assessing the three poetry datasets above, we also evaluated the example quatrains in [2; 13; 15; 16; 17; 18; 19] using the text-fluency-checker and the tone-checker. Although the number of example poems shown above is 1, 2 or 3 at most, it still has some reference values. The average scores are shown in Table 2.  
The average fluency-checking score from our CCPTXL model was higher than from CCPSA. Furthermore, it also obtained a higher score than these randomly chosen poems from CCPD. SPG [19] and CAE-HD [17] obtained high scores on fluency but they are only quatrains, whereas our generated regulated verses are twice as long as theirs. This demonstrates that our poem model can generate more fluent and coherent longer sentences. The distribution of the poems of each data set on the three levels is shown in Figure 4. Among the generated 1000 poems from our model, 98.3% were evaluated as Level 0, 1.7% as Level1, and no poems were Level 2. This shows that our model can stably compose high-quality poems. The tone-checking scores of poems were calculated by the tone-checker, in which each poem is checked and scored according to the tone and rhyming rules. Our CCPTXL model obtained the highest score of 10, as shown in in Table 2. This shows that almost all the poems generated by our model can obey the tone and rhyming rules of regulated verse. Figure 5 shows that the poems composed by our CCPTXL model scored 9-10 and 8-9, accounting for 89.5% and 7.9%, respectively.