Using Multi-Encoder Fusion Strategies to Improve Personalized Response Selection (2022)

Abstract: Personalized response selection systems are generally grounded on persona. However, a correlation exists between persona and empathy, which these systems do not explore well. Also, when a contradictory or off-topic response is selected, faithfulness to the conversation context plunges. This paper attempts to address these issues by proposing a suite of fusion strategies that capture the interaction between persona, emotion, and entailment information of the utterances. Ablation studies on the Persona-Chat dataset show that incorporating emotion and entailment improves the accuracy of response selection. We combine our fusion strategies and concept-flow encoding to train a BERT-based model which outperforms the previous methods by margins larger than 2.3% on original personas and 1.9% on revised personas in terms of hits@1 (top-1 accuracy), achieving a new state-of-the-art performance on the Persona-Chat dataset

## introduction currently, most response selection systems tend to perform well in most cases @xcite @xcite . however, these re-ranking systems have the poor capability to detect and evade contradictory responses. responses selected by these systems often contradict previous utterances, and any form of contradiction may disrupt the flow of conversation. previous research has attempted to incorporate persona while selecting @xcite or generating @xcite responses to maintain consistency. additionally, a correlation exists between persona with personality (leary and allen, 2011) , which influences empathy @xcite . zhong et al. presented a multi-domain dataset collected from several empathetic reddit threads contributing towards personabased empathetic conversations. nevertheless, no one has studied the emotion-persona interplay in data presented in a more natural form. figure 1 depicts situational emotion sometimes needs more preference than the chatbot's persona in response selection. on the contrary, different personality traits are related to emotion regulation difficulties @xcite . due to this, a person's expected emotions can deviate based on his persona. besides that, we also observe that concepts discussed in a conversational flow play an important role in response selection. however, no one has incorporated this in response selection. model emotion inappropriate(%) contradictory(%) off-topic(%) bert-cra 7.35 11.88 12.3 we can infer the significance of these problems from subsequently, we propose three fusion strategies, emotion-aware(ema), entailmentaware(ena), persona-entailment-aware(p-ema). these fusion strategies are designed based on emotion-persona interaction or persona-entailment information. we propose a concept-flow encoding technique that matches relevant concepts from the context and candidate responses with these fusion strategies. we test our proposed methods on the persona-chat dataset with our automatic annotation. the results show that a model trained on a combination of our proposed fusion strategies outperforms the current state-of-the-art model by a margin of 2.3% in terms of top-1 accuracy hits@1. in summary, the contributions of this paper are three-fold. (1)automatically annotate persona-chat dataset with utterance level emotion, entailment, and concept information to provide extra supervision. (2) a suite of fusion strategies and a concept-flow encoder which are designed and implemented into a series of models, aiming to explore the impact of emotion, entailment, and concept-flow in the task of response selection. (3) experimental results demonstrate that our proposed models outperform the existing state-of-the-art models by significant margins on the widely used persona-chat response selection benchmark. ## dataset in this work, we extend persona-chat @xcite and augment it with a series of annotators. the dataset consists of 8939 complete dialogues for training, 1000 for validation, and 968 for testing. responses are selected at every turn of a conversation sequence, resulting in 65719 context-responses pairs for training, 7801 for validation, and 7512 for testing. the positive and negative response ratio is 1:19 in the training, validation, and testing sets. there are 955 possible personas for training, 100 for validation, and 100 for testing, consisting of 3 to 5 profile sentences. a revised version of persona descriptions is also provided by rephrasing, generalizing, or specializing the original ones to make this task more challenging. ## automatic dataset annotation we have annotated the persona-chat with the help of a series of automatic annotation schemes. since we are studying the effect of emotions in personalized response selection, we assign emotion labels to the personas, context-utterances, and candidate responses using an emotion classifier. personas and utterances were annotated using an entailment classifier to incorporate the entailment information while selecting responses. finally, we follow a multi-layer keyword mining strategy to match meaningful concepts appearing in the context and response. ## conclusion this work proposes a suite of novel fusion strategies and concept-flow encoder, which leverages the utterances' emotion, entailment, and concept information. these features help improve the performances of our models and provide critical insights into certain aspects of how humans communicate with each other. though the techniques used in this paper are simple, it highlights the areas where response selection often falters, like detecting contraction, deviation from the concepts, etc. this work can be further extended by using a graphical