Towards Robust Personalized Dialogue Generation via Order-Insensitive Representation Regularization (2023)

Abstract: Generating persona consistent dialogue response is important for developing an intelligent conversational agent. Recent works typically fine-tune large-scale pre-trained models on this task by concatenating persona texts and dialogue history as a single input sequence to generate the target response. While simple and effective, our analysis shows that this popular practice is seriously affected by order sensitivity where different input orders of persona sentences significantly impact the quality and consistency of generated response, resulting in severe performance fluctuations (i.e., 29.4% on GPT2 and 83.2% on BART). To mitigate the order sensitivity problem, we propose a model-agnostic framework, ORder Insensitive Generation (ORIG), which enables dialogue models to learn robust representation under different persona orders and improve the consistency of response generation. Experiments on the Persona-Chat dataset justify the effectiveness and superiority of our method with two dominant pre-trained models (GPT2 and BART).

## introduction developing a persona-consistent dialogue model has been one of the key issues and crucial problems in open-domain dialogue systems @xcite . @xcite define the problem of personalized dialogue generation, which aims to generate personalized responses based on textually described persona profiles. many efforts have been made on developing dialogue models that generate responses consistent with the provided persona profile @xcite @xcite . the recent development in transformer-based pre-trained models @xcite @xcite ) has led to great successes in dialogue systems @xcite , 2019; @xcite @xcite @xcite . inspired by these successes, previous works incorporate those pre-trained models in persona-based response generation by concatenating the dialogue history and persona as input to generate the response in an auto-regressive manner @xcite . however, a fine-tuned model can generate a high-quality and persona-consistent response in a certain ordering of personas, while varying this order may lead to a generic and even inconsistent response as illustrated by the example in figure 1 . we empirically show that the worst ordering of persona can lead to a 29.4% decline in bleu score compared with the best ordering. ideally, a well-trained dialogue generation model should be able to generate a persona-consistent response regardless of the ordering of personas in the input. we perform experiments and analyses to identify the cause of the ordering sensitivity. we find that the ordering of persona in the input leads to different representations of context and response. we also show that the model can attend to the appropriate persona and generate high-quality responses under some representations but not under others. ## related work maintaining a consistent persona is essential for building a human-like dialogue system, where most works regard persona as a set of sentences along with each dialog @xcite @xcite @xcite . @xcite disentangled the task of persona-based dialogue generation into two sub-tasks: consistency understanding and dialogue generation while @xcite aims to alleviate the problem of limited data by data manipulation methods. despite satisfactory performance in previous work, the impacts of different orders of personas are still under-explored, resulting in unstable and inconsistent responses. our work is also related to work on order sensitivity in prompt-based few-shot learning @xcite . @xcite found that the different order of training examples in the prompt can cause accuracy to vary from near chance to state-of-the-art in the few-shot classification setting. similarly, order sensitivity for in-context learning also exists regardless of model size and the prompt format @xcite . distinguishing from them, we focus on order sensitivity in the language generation task in finetuning setting, especially the impacts of persona orderings to generate persona-consistent responses. ## order sensitivity problem and analysis in this section, we first illustrate the seriousness of the order sensitivity problem by showing a huge performance fluctuation in persona dialogue models when fed the same personas in the best and worst orders. then we analyse why their performance is volatile to different persona orderings. to illustrate the problem, we finetune plms on the persona-chat by concatenating the persona and dialogue context together to predict the target response, including gpt2 and bart. after the training converges, we test them on two settings: (1) the best case: for each test sample, we feed the models all possible permutations of persona sentences and keep the maximum score for each sample as the final score; (2) the worst-case: perform the same process as (1), but take the minimum score. ## method we introduce the proposed framework, named orig: order insensitive generation (orig). as shown in figure 2 , we transform the persona ordersensitivity problem as a constrained optimization problem that optimises a persona dialogue model under the uncertainty of the input persona order. ## conclusion we show that the current practice of applying pretrained models to the personalized dialogue generation task is volatile across different input orders of personas. through the analysis, we find that the problem arises from the representation changes induced by the input changes. motivated by these, we propose our orig, a model-agnostic framework for finetuning the persona dialogue model such that it obtains a persona order-invariant representation. experiments on two dominant pre-trained dialogue models show that our framework improves performance and reduces order volatility.