Partner Personas Generation for Dialogue Response Generation (2022)

Abstract: Incorporating personas information allows diverse and engaging responses in dialogue response generation. Unfortunately, prior works have primarily focused on self personas and have overlooked the value of partner personas. Moreover, in practical applications, the availability of the gold partner personas is often not the case. This paper attempts to tackle these issues by offering a novel framework that leverages automatic partner personas generation to enhance the succeeding dialogue response generation. Our framework employs reinforcement learning with a dedicatedly designed critic network for reward judgement. Experimental results from automatic and human evaluations indicate that our framework is capable of generating relevant, interesting, coherent and informative partner personas, even compared to the ground truth partner personas. This enhances the succeeding dialogue response generation, which surpasses our competitive baselines that condition on the ground truth partner personas.

## introduction building informative and engaging dialogue agents @xcite is a popular research direction within the area of natural language processing. for the sake of engagement, diverse and consistent responses @xcite are important factors, and personas information @xcite gives rise to both. there are two types of personas, namely self persona and partner persona. the former refers to a self profile consisting of several sentences representing the dialogue agents. such a persona allows producing consistent responses rather than solely relying on the personas that are randomly learned and embedded in the model parameters @xcite . the latter refers to a profile that represents the users. leveraging such partner personas has been empirically shown to be helpful for dialogue response selection @xcite . unfortunately, the existence of partner personas suffers from the cold start @xcite @xcite at the beginning of the conversation. most of the works, if not all, @xcite @xcite @xcite @xcite @xcite have been either overlooking partner personas or simply focusing on the impractical situation where partner personas guarantee to exist. in contrast, our work does not suffer from the practical issue when partner personas are missing during inference, and our proposed framework surpasses the baseline that conditions on the ground truth partner personas. to our knowledge, this is the first attempt to formulate partner personas generation for improved performance on the downstream dialogue response generation. our work is motivated by the underlying hypothesis that partner personas generation is plausible given the self personas and dialogue context. automatic and human evaluation results support the hypothesis and indicate that generated personas are even more interesting than the ground truth, which improves the downstream dialogue response generation. this paper thus paves the way to exploit partner personas generation (ppg) for dialogue response generation @xcite . we propose a novel framework composed of three major components, namely a personas generator, a dialogue response generator and a critic network. the personas generator generates partner personas, which the dialogue response generator conditions on. we employ reinforcement learning with a critic network that propagates the reward back to the generators for joint training. prior works have investigated partner persona retrieval @xcite . the human-constructed ground truth personas serve as the upper bound for such retrieval-based systems, and we argue that the ground truth is not coherent and diverse enough. interestingly, we observe that the generative counterpart proposed in our framework generates relevant, informative and coherent partner personas, which further improves the succeeding dialogue response generation. it follows another advantage that our framework does not need an external database to retrieve from @xcite . one close work to ours is a multi-task framework for meta-learning (lee et al., 2021) that uses personas reconstruction as an auxiliary task to improve response consistency. the differences are that theirs does not differentiate between self personas and partner personas, while ours does. theirs indicates an improvement over personality consistency, while ours report improvements for the overall quality. we conduct an empirical comparison with their model by reconstructing the partner personas. experimental results indicate that such a multi-task model does not work well in our problem setting. very recently, @xcite formulates personas generation as a seq2seq task for improved downstream response generation via multi-task learning. in contrast, our work leverages reinforcement learning to jointly train the partner personas generator and the response generator. automatic and human evaluation results indicate that our framework can generate partner personas that are more diverse and interesting than the ground truth partner personas and generate more diverse and engaging responses than the baseline conditioned on ground truth partner personas. foot_0 2 related work ## proposed framework we propose a novel framework composed of three major components, namely a partner personas generator, a dialogue response generator and a reinforcement learning component with a critic network. figure 1 depicts the inference flow of our setting. the input dialogue context with self persona is first fed into the partner personas generator. the generated partner personas output is then concatenated with the dialogue context and the self personas as the input into the dialogue response generator. in the beginning, we train our partner personas generator and dialogue response generator under supervised learning. in the training stage, we use the ground truth partner personas to train the dialogue response generator, and we replace it with generated partner personas in the inference stage. after the supervised learning stage, the second stage is a reinforcement learning stage which jointly optimizes both partner personas generator and dialogue response generator as depicted in figure 2 to train the partner personas generator under the reward signal that is relevant to dialogue response generation as well as fine-tuning dialogue response generator trained on the generated partner personas. 2 particularly, we employ a dedicatedly designed critic network that receives generated partner personas and generated dialogue responses as the input and output a reward that measures the relevance between the generated personas and responses and propagates back to the generators. ## conclusion our novel framework incorporates partner personas generation into dialogue response generation. it effectively mitigates the problem that partner personas are not available in practical applications as well as the cold start problem during early conversation. the experimental results with both automatic and human evaluation demonstrate that our framework generates coherent, diverse, interesting and engaging partner personas, even compared to the ground truth partner personas. we employ reinforcement learning with a dedicatedly designed critic network that boosts the response generation by conditioning on the generated personas. automatic and human evaluation results indicate that our response generator surpasses our competitive baselines that condition on the ground truth partner personas. extensive case studies demonstrate that our framework can generate satisfying dialogue responses and partner personas.