# A Personalized Multi-Turn Generation-Based Chatbot with Various-Persona-Distribution Data
## 5 Result
### Further Analysis  
#### 5.2.1 Ablation Study  
As stated above, our model performs well, and we analyzed it in four aspects in the model: (1) adapted persona, (2) consistency inference, (3) dual decoder, and (4) multi-turn context. We design a ablation experiment, as in Tables 5 and 6, as follows: **w/o UL\({}_{\bf D_{1}}\)** the model removes the persona selection auxiliary task; **w/o UL\({}_{\bf D_{2}}\)** the model removes the consistency inference auxiliary task; **w/o UL\({}_{\bf D_{1}}\)&UL\({}_{\bf D_{2}}\)** the model removes both auxiliary tasks; **w/o D\({}_{\bf 2}\)** the model removes the decoder \(D_{2}\), and since consistency inference is acted on \(D_{2}\), it is removed as well; \(\mathbf{w/E+D1}\) the model removes the decoder \(D_{2}\) and both auxiliary tasks, and the model degenerates to a transformer model.  
**The effect of adapted persona**: Adapted personas in our model are predicted by the classifier \(C_{ps}\) from the given personas and are consistent with the speaking style implied in the context. The adapted persona representations are fused into the original response \(R_{1}\) through attention routing and trained with a persona-selection auxiliary task. As shown in Tables 5 and 6, comparing the full model with **w/o UL\({}_{\bf D_{1}}\)**,**w/o UL\({}_{\bf D_{2}}\)**, and **w/ E+D1**, we can see that the incorporation of adapted persona has a significant improvement on Per.Acc in both datasets, especially in PersonaChat. The explanation is that more responses are persona-relevant in the persona-dense dataset, and the upper limit of the persona distribution is higher.  
**The effect of consistency inference**: The consistency inference task works as an auxiliary task of decoder \(D_{2}\) to reduce inconsistent persona information in the rewritten responses. Compared with the full model, the **w/o UL\({}_{\bf D_{2}}\)** experiments have a significant decrease in \(\Delta\)P and manually evaluated consistency in Tables 5 and 6. We can assume that the consistency inference task is key to the semantic consistency of personas.  
**The effect of dual decoder**: The dialogue generation process in the model consists of two individual decoders. There are two doubts about the dual decoder: (1) Does rewriting the decoder have an effect? (2) Is the improvement in the model just the result of a larger-scale model? For question (1), we compare **w/o UL\({}_{\bf D_{2}}\)** with **w/o D\({}_{2}\)** and find that decoder \(D_{2}\) shows a vast improvement in perplexity, distinct 1/2, and manually evaluated fluency and diversity, and we determine that the larger model has a better performance on language understanding. For question (2), the larger model scale is not the only reason for the good performance of our model. Firstly, we compare our models, as shown in Tables 5 and 6, and find that the full model has a significant improvement in all aspects than **w/o UL\({}_{\bf D_{1}}\)and UL\({}_{\bf D_{2}}\)**, which confirms that the two auxiliary tasks we proposed contribute to all metrics. In addition, compared with a model with a dual-decoder structure, BoB, persona selection task, and attention routing significantly contribute to our model in Tables 3 and 4.  
**The effect of multi-turn context**: Since persona selection in our model refers to context, we chose multi-turn context for our inputs instead of single-turn queries. To study the effect of multi-turn contexts on our model, we conducted a \(\mathbf{w/E+D1}\) experiment, where the model differs from the transformer model only in the input. Comparing the results of the two models, we find that the results of \(\mathbf{w/E+D1}\) outperform transformer on all metrics in PersonaChat, in Table 5. However, the results are opposite on the PersonalDialog in Table 6. We guess that persona information hinders language understanding on sparse datasets and affects the quality of the generated responses. This is why we use the persona selection task to focus on persona distribution.