Toward Stance-based Personas for Opinionated Dialogues (2020)

Abstract: In the context of chit-chat dialogues it has been shown that endowing systems with a persona profile is important to produce more coherent and meaningful conversations. Still, the representation of such personas has thus far been limited to a fact-based representation (e.g. “I have two cats.”). We argue that these representations remain superficial w.r.t. the complexity of human personality. In this work, we propose to make a step forward and investigate stance-based persona, trying to grasp more profound characteristics, such as opinions, values, and beliefs to drive language generation. To this end, we introduce a novel dataset allowing to explore different stance-based persona representations and their impact on claim generation, showing that they are able to grasp abstract and profound aspects of the author persona.

## introduction while chit-chat neural models have obtained impressive improvements in recent years, they are known to suffer from key limitations: they tend to lack specificity and to lose coherence as the conversation unfolds, becoming less captivating. one explanation is that they do not have a consistent personality; for this reason, some approaches proposed to explicitly encode the persona via a small set of claims describing the characteristics of the agent, such as "my dad has a car dealership", "i have two cats" @xcite . such representations provide a fact-based background context useful to drive and ground the relevance of the conversational acts for the dialogue at hand, but with little generalization capability. pushing this approach a step beyond, we thus investigate the construction of stance-based personas, in order to grasp profound and intimate characteristics -such as opinions, values, and beliefs. this could allow agents to sustain personal points of view both within the same conversation and across different discussions. in this paper, we make a first attempt at representing persona with different approaches and levels of abstraction. we build a new conversational dataset from a social platform dedicated to argumentative interaction, foot_0 and report experiments for stance-based personas with varying degrees of abstraction (e.g. implicit and explicit stance representation). our experiments show that stancebased personas enable the agents to intervene, consistently with their representation, across topics unseen at training time. the advantage of pre-trained, transformer-based, language models is that they can capture long-term dependencies and generate texts that are fluent, varied, and rich in content, mitigating many of the limitations of previous neural dialogue models, such as contents inconsistency @xcite et al., 2019a; gao et al., 2018, 2019), lack of longterm contextual coherence (serban et al., 2017), and blandness @xcite @xcite . persona approaches were recently developed with the introduction of end-to-end dialog system based on memory networks, which allow to encode the persona profile as a simple list of statements. one of the first datasets specifically developed for persona-based dialogues was released by @xcite . another approach consists in modeling a system persona in terms of interaction style (e.g. formal vs. informal register) as used in goal-oriented settings by @xcite ; @xcite to provide personalized interactions. further, @xcite showed how injecting these specific persona-related aspects into a conversation can positively affect the interaction in goal-oriented scenarios, both in terms of quality of service and overall perceived quality. ## the kialo dataset the construction of a stance-based persona requires a deeper peek over the opinions, beliefs, and stances of an author, expressed through textual claims possibly across different topics. to this end, turning to transactional crowd-sourcing approaches is in our opinion not ideal: asking crowd-workers to publish private opinions is ethically questionable, while inducing them to engage meaningfully across several topics poses challenges from a design perspective. last but not least, collecting a significantly sized dataset would require a consistent budget that can easily amount to hundreds of thousand dollars. 2 for these reasons we turned our attention to kialo, a public discussion platform letting its users debate in a constructive and rational way with peers. issues to philosophy, religion or even science fiction. all these elements make it the ideal resource for our goals. in kialo, the users can easily inspect every aspect and claim of a discussion through a tree-shaped structured visualization and decide where to intervene. in this tree, the top node is defined as the thesis claim and each claim in the tree supports or opposes its parent claim, i.e. pro or con. an example discussion is shown in figure 1 . we have collected 1,580 english discussions and 241,882 unique claims in these discussions. 3 the number of unique claims in the collected discussions varies widely (µ = 153.08, σ = 269.58), as does their depth (µ = 6.31, σ = 4.79). considering the structure of the discussions in kialo, each sample in the dataset we collected is composed by author id, claim id, claim, stance label, parent id, and parent claim. in this respect, the instances in the dataset are similar to single-turn dialogues. for our experiments below, we sampled 5% of discussions for the test and 5% for the validation sets, resulting in 79 discussions for validation, 79 for test, and 1,422 for training. the sampling has been conducted in a stratified fashion according to the number of the claims in each discussion. ## model we frame our problem as a text generation task, where the probability to generate a sequence y composed of n tokens, y 0 , ..., y n , is given by: ## human evaluation to get a deeper understanding of our models, we also run a human assessment of the outputs generated by the following model configurations, compared to the ground truth: i) the baseline model that has only access to parent claim, without any persona; ii) p exp,dynamic , trained with parent claims and the explicit, dynamically selected, persona; and iii) p exp,hybrid , also trained with parent claims and the explicit, dynamically selected, persona, but fed at inference time with a dynamic selection of the persona (corresponding to the last row in evaluation protocol to evaluate each generated output w.r.t. the author persona, it is important to chose a neutral representation of this persona, so to avoid favoring any model and biasing the human evaluation. we decided to use p imp , the implicit persona, which we believe is the most neutral amongst the 4 models we evaluate. we randomly sampled 50 claims from the test set, under the constraint that the corresponding authors had provided at least 10 claims to the training set. the pool of eligible claims under such criterium compounds to 10,995 (out of the 11,689 in the test set) from 1,251 different authors. this ensures that a large persona representation can be built for all the selected samples. we asked three professional english speakers to score their relevance towards the implicit persona and the parent claim, on a likert scale ranging from 1 to 5. to assess relevance, the annotators were presented only with the sample to evaluate, paired with either the corresponding parent claim or the associated implicit persona. ## conclusions endowing dialogue agents with persona profiles is important to produce more coherent and meaningful conversations. in particular, we argue for using stance-based personas to drive language generation consistently with profound characteristics -such as opinions, values, and beliefs. to this end, we introduced a novel dataset and explored diverse stance-based persona representations and their impact on claim generation. in future works, we plan to enrich the persona representation with additional information available in kialo (e.g. authors' votes to others claims), to encode more complex profiles; further, we will extend the presented approach to multi-turn interactions, as enabled by the kialo discussions structure.