Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity (2022)

Abstract: State-of-the-art dialogue models still often stumble with regards to factual accuracy and self-contradiction. Anecdotally, they have been observed to fail to maintain character identity throughout discourse; and more specifically, may take on the role of their interlocutor. In this work we formalize and quantify this deficiency, and show experimentally through human evaluations that this is indeed a problem. In contrast, we show that discriminative models trained specifically to recognize who is speaking can perform well; and further, these can be used as automated metrics. Finally, we evaluate a wide variety of mitigation methods, including changes to model architecture, training protocol, and decoding strategy. Our best models reduce mistaken identity issues by nearly 65% according to human annotators, while simultaneously improving engagingness. Despite these results, we find that maintaining character identity still remains a challenging problem.

## introduction the exchange of stories from one's past, or descriptions of activities in one's present, are a fundamental part of human discourse. trustworthy human conversationalists keep their stories roughly straight within a conversation. an interlocutor taking on your own stories and persona as theirs is especially jarring and unnatural. however, despite the improvements in state-of-the-art open-domain dialogue modeling, both in terms of distributional accuracy metrics like perplexity, and subjectively in terms of human judgements @xcite , interactions with those agents reveal that they cannot keep their stories straight. in particular, they are likely to take on the role of their interlocutor; for example, if an agent's partner says they are a software engineer, the agent is likely to say it is a software engineer too @xcite , or worse, appropriate their partners just told tale of a trip to naacl as their own. some setting: barn, farm partner: thief self: guest guest: hail who goes there?! thief: hello friend i mean you no harm model response guest: i am not your friend, i am your thief! i am here to steal from you! character token probabilities thief: 0.39 guest: 0.23 setting: castle kitchens, inside palace partner: hunter self: helper hunter: hi helper: hello. what brings you to the kitchen? hunter: i am chopping some meat that is to be sold. helper: could you spare me a piece? i'm starving. hunter: it's 3 cents a pound, if you have 3 cents then yeah model response helper: i don't have any money. i'm just a poor hunter trying to make a living. character token probabilities hunter: 0.39 helper: 0.12 in this work we formalize and quantify this behavior, show that to some extent it can be detected automatically with a specifically trained classifier, and then study a wide variety of mitigations. these include multi-objective training, unlikelihood training @xcite , classifier-assisted re-ranking based generation, and several forms modifying the attention mechanisms of the decoder in a sequence to sequence model. our best methods can reduce mistaken identity issues by 65%, while simultaneously improving inconversation engagingness; indeed, our models that can stick to their role in conversation are judged by humans to be significantly more engaging than their baseline counterparts. despite these advances, we find that there is still considerable space to improve these results further in future work. we make publicly available both our trained models and code to reproduce results 1 . ## conclusion in this work we explored the problem of maintaining one's character in open dialogue, and showed that state-art-of-the-art models have a fundamental weakness in this regard. we provided a clear framing of the problem and showed one can build automatic metrics (rpa) that evaluate models using a classifier. we then explored a variety of methods throughout this paper. while a wide variety of well-known techniques, such as multi-objective or unlikelihood training, have little impact, we found that expanded attention and re-ranking are two approaches that can help to a degree, and their combination also improves results. our introduced method pacer performs well and may be suitable for other tasks beyond the focus of this paper. nevertheless, our best methods still lag behind human (crowdworker) performance in several regards, e.g. 1.34% vs. 2.23% in terms of mistaken identity per turn, or 5% vs. 14.7% per conversation. therefore considerable progress still has to be made on this challenging problem.