# Research on Climactic Chapter Recognition of a Chinese Long Novel Based on Plot Description
## 4 Experimental Design and Analysis
### Experimental Result and Analysis  
Comparison and ablation experiments are conducted to verify the effectiveness of the proposed model CCRPCM. The models used in the comparison experiments are NB [14], SVM [12], BiLSTM [21], and the pre-training model Roberta-large [22]. The ablation models include CCRPCM without the Plot Description Matrix (CCRPCM-PCM), CCRPCM without the Chapter Plot Difference Matrix (CCRPCM-PDM), CCRPCM without BiGRU (CCRPCM-BiGRU), and CCRPCM without the Multi-Attention Mechanism (CCRPCM-MultiAtt). The experimental results are shown in Table 5 (The bold font indicate the best experimental results). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Parameter** & **Denotation** & **CCRPCM** & **BiLSTM** & **Roberta-Large** \\ \hline max\_epoch & maximum number of iterations & 128 & 128 & 128 \\ batch\_size & batch size & 4 & 16 & 16 \\ lr & initial learning rate & 0.0001 & 0.0001 & 0.0002 \\ hidden\_size & dimension of the hidden state & 256 & 256 & 1024 \\ dropout & loss rate & 0.1 & 0.1 & 0.2 \\ num\_heads & number of heads in attention & 8 & - & 16 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Parameter setting. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multicolumn{1}{c}{\multirow{2}{*}{**The Legend of the Condor Heroes**}} & \multicolumn{6}{c}{**“The Condor Heroes Return”**} & \multicolumn{6}{c}{**“Heaven Sword and Dragon**} \\ \multicolumn{1}{c}{} & \multicolumn{2}{c}{**Heroes”**} & \multicolumn{2}{c}{**[PERSON]”**} \\ \hline
**Model** & **R** & **P** & **F1** & **R** & **P** & **F1** & **R** & **P** & **F1** \\ \hline NB & 53.28 & 51.02 & 52.13 & 53.96 & 52.80 & 53.37 & 56.10 & 55.70 & 55.90 \\ SVM & 57.07 & 60.56 & 58.76 & 55.68 & 55.49 & 55.33 & 59.17 & 60.00 & 59.58 \\ Roberta-large & 67.89 & 69.87 & 68.87 & 64.45 & 63.86 & 64.16 & 64.54 & 64.11 & 64.33 \\ BiLSTM & 66.18 & 72.11 & 68.20 & 68.20 & 67.85 & 68.76 & 69.48 & 70.24 & 69.81 \\ CCRPCM-PCM & 60.43 & 61.16 & 60.79 & 64.82 & 65.46 & 65.12 & 67.53 & 69.27 & 68.17 \\ CCRPCM-PDM & 51.67 & 52.16 & 51.45 & 51.43 & 51.08 & 50.98 & 57.84 & 58.33 & 58.03 \\ CCRPCM-BiGRU & 49.51 & 49.57 & 49.48 & 48.66 & 46.54 & 46.67 & 47.14 & 48.33 & 46.67 \\ CCRPCM-MultiAtt & 72.50 & 70.17 & 71.32 & 75.00 & 73.05 & 74.01 & 67.50 & 72.25 & 69.79 \\ CCRPCM & **80.00** & **75.68** & **77.78** & **78.95** & **83.33** & **81.08** & **71.37** & **75.00** & **72.54** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Experimental results (U.%). Based on the experimental corpus of \"The Legend of the Condor Heroes\", CCRPCM has the best performance followed by BiLSTM, while NB is the worst. The F1 values of CCRPCM are 25.65%, 19.02%, 8.91%, and 9.58% higher than those of NB, SVM, Roberta-large, and BiLSTM respectively. Based on the experimental corpus of \"The Condor Heroes Return\", CCRPCM also has the best performance followed by BiLSTM, with NB performing the worst. The F1 values of CCRPCM are 27.71%, 25.75%, 16.92%, and 12.32% higher than those of NB, SVM, Roberta-large, and BiLSTM, respectively. The result is similar based on the experimental corpus of \"Heaven Sword and Dragon Sabre\", where \(F1\) values of CCRPCM are 16.64%, 12.96%, 8.21%, and 2.73% higher than those of NB, SVM, Roberta-large, and BiLSTM, respectively. It can be concluded from the above experimental results that CCRPCM has better recognition performances than the other models, such as NB, SVM, Roberta-large, and BiLSTM. The result of BiLSTM is closer to that of Roberta-large, while NB and SVM do not perform well. The main reasons are listed as follows. 1. NB assumes that the features are independent of each other, but actually there is a high correlation between the features of the novel chapters. This leads to its poor performance. Moreover, this model requires prior probabilities in advance that generally depends on assumptions, which further increases the risk of misclassification. 2.