# A Personalized Multi-Turn Generation-Based Chatbot with Various-Persona-Distribution Data
## 5 Result
### Model Performance  
As shown in Tables 3 and 4, our proposed model obtained better performance on all automatic and manual metrics than other models on both datasets, indicating that our model effectively utilizes persona information to generate persona-adapted and persona-consistent high-quality responses. Specifically, on the conversation generation task, compared to the large-scale pre-trained model OPT, our models achieve better fluency and diversity. This result is attributed to the framework of the dual decoder and the attention-routing mechanism, because BoB, which uses the dual decoder, and AR, which uses attentional routing, do not perform better than OPT. Unlike BoB, AR uses a single-sentence query as input. Another reason is that our model learns from multi-turn context and can obtain sentence-level hidden vectors to improve fluency. In addition, we observe that DialogWAE and PerCVAE have an obvious gap with other baselines. This demonstrates that the attention mechanism of the large-scale pre-trained model has a substantial enhancement for the conversation generation task. Benefiting from the transformer structure, our model has a massive advantage in language understanding over earlier models. Comparing the performance of PerCVAE and DialogWAE, which are similar in structure, we observe two phenomena: 1. The responses of PerCVAE contain more persona information, which demonstrates the effectiveness of the fused persona information process. 2. PerCVAE has no significant improvement in persona consistency understanding, demonstrating the necessity to constrain the process of fusing persona information. Moreover, our model and BoB achieve the top two on the consistency metric, which shows that the process of fusing persona information should be constrained. Since our proposed persona selection \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline
**Model** & **ppl.** & **Dist.1** & **Dist.2** & **p.Ent** & **p.Ctd** & \(\Delta\)P & **C.Score** & **Per.Acc** & **Fluency** & **Diversity** & **Consistency** \\ \hline Ours & 12.9 & 6.17 & 23.70 & 10.8 & 93.6 & 82.8 & 4.19 & 3.84 & 3.59 & 3.21 & 3.62 \\ w/o UL\({}_{D_{1}}\) & 13.8 & 5.75 & 21.41 & 11.3 & 78.6 & 67.3 & 3.93 & 1.73 & 3.38 & 3.19 & 3.26 \\ w/o UL\({}_{D_{2}}\) & 14.7 & 5.61 & 20.37 & 12.9 & 26.3 & 13.4 & 2.17 & 3.04 & 3.47 & 3.03 & 3.31 \\ w/o UL\({}_{D_{1}}\)\&UL\({}_{D_{2}}\) & 15.3 & 5.64 & 19.70 & 13.0 & 24.1 & 11.1 & 2.01 & 1.69 & 3.26 & 3.14 & 3.08 \\ w/o \(D_{2}\) & 38.9 & 2.03 & 9.74 & 35.8 & 41.7 & 5.9 & 1.48 & 2.95 & 2.84 & 2.93 & 2.53 \\ E+D1 & 43.4 & 1.66 & 8.31 & 41.2 & 43.4 & 2.2 & 1.39 & 1.26 & 2.73 & 2.95 & 2.04 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Ablation study results of PersonalDialog dataset. \begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline
**Model** & **ppl.** & **Dist.1** & **Dist.2** & **p.Ent** & **p.Ctd** & \(\Delta\)P & **C.Score** & **Per.Acc** & **Fluency** & **Diversity** & **Consistency** \\ \hline DialogWAE [32] & 52.4 & 2.52 & 8.92 & 50.2 & 54.9 & 4.7 & -1.95 & 0.16 & 2.57 & 2.26 & 2.07 \\ GPT2 [33] & 24.0 & 4.67 & 16.59 & 23.5 & 49.8 & 26.3 & 2.47 & 1.67 & 2.98 & 2.75 & 3.04 \\ OPT [34] & 16.8 & 5.82 & 18.52 & 15.4 & 67.3 & 52.3 & 3.83 & 1.93 & 3.44 & 2.93 & 3.42 \\ PerCVAE [4] & 58.5 & 2.38 & 7.53 & 52.3 & 65.2 & 12.9 & 0.81 & 0.35 & 2.21 & 2.07 & 2.35 \\ Transformer [29] & 39.1 & 2.82 & 11.58 & 38.4 & 44.1 & 5.7 & 0.54 & 0.28 & 2.63 & 2.36 & 2.39 \\ AR [18] & 32.6 & 4.45 & 15.96 & 27.7 & 56.2 & 28.5 & 1.32 & 0.91 & 2.65 & 2.58 & 2.78 \\ BoB [9] & 17.4 & 5.62 & 18.31 & 15.2 & 72.5 & 57.3 & 3.51 & 1.27 & 3.04 & 2.85 & 3.34 \\ Ours & **12.9** & **6.17** & **23.70** & **10.8** & **93.6** & **82.8** & **4.19** & **2.84** & **3.59** & **3.21** & **3.62** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Experiment results of PersonalDialog dataset.