Opening up Minds with Argumentative Dialogues (2022)

Abstract: Recent research on argumentative dialogues has focused on persuading people to take some action, changing their stance on the topic of discussion, or winning debates. In this work, we focus on argumentative dialogues that aim to open up (rather than change) people’s minds to help them become more understanding to views that are unfamiliar or in opposition to their own convictions. To this end, we present a dataset of 183 argumentative dialogues about 3 controversial topics: veganism, Brexit and COVID-19 vaccination. The dialogues were collected using the Wizard of Oz approach, where wizards leverage a knowledge-base of arguments to converse with participants. Open-mindedness is measured before and after engaging in the dialogue using a questionnaire from the psychology literature, and success of the dialogue is measured as the change in the participant’s stance towards those who hold opinions different to theirs. We evaluate two dialogue models: a Wikipedia-based and an argument-based model. We show that while both models perform closely in terms of opening up minds, the argument-based model is significantly better on other dialogue properties such as engagement and clarity.

## introduction developing dialogue agents that are able to argue about different topics has been the focus of a lot of recent research. typically, these agents engage in conversations with people with the aim of changing their opinions on a topic or winning debates. accordingly, success of argumentative dialogue agents has been measured by their ability to convince people to take an action such as donating to a charity @xcite , change their position on the subject of discussion @xcite , or attract more votes by the audience listening to their debates @xcite . other work has studied argumentation with p: i feel like pushing your kids to not eat meat or dairy products is a bit too much, other than that i really don't see a problem if it's someones choice to not eat meat or dairy products. w: the same goes for animal products, where they should not be forced onto children until they're old enough to decide for themselves. p: if a child chooses to eat meat in a vegan household they should absolutely be allowed to and have that privilege. w: that is fair. however, children are often not educated on the source of their food and distracted from their natural empathy for animals. the artificial divide between animals which are okay to eat and those that are not begins with parenting. figure 1 : a dialogue excerpt from our dataset about veganism between a participant (p) and a wizard (w). the aim of reaching agreement @xcite . nonetheless, none of the previous works has studied dialogues in terms of their ability to stimulate open-minded thinking and help participants learn about views that are unfamiliar or in opposition to their own and become more tolerant towards people who hold these views. open-minded thinking has been motivated by many psychological studies. @xcite showed that it correlates with information acquisition. @xcite demonstrated its importance for responsible behaviour on social media platforms. more recently, @xcite suggested that individual's negative views about their ideological opponents could partly be attributed to their lack of exposure to good arguments for these views. motivated by research on open-minded thinking, we propose to use argumentative dialogues to expose participants to different opinions about polarising topics with the aim of opening up their minds and increasing their tolerance towards views opposing their own. we collected 183 dialogues about three controversial topics @xcite , using the wizard of oz (woz) approach @xcite . the wizards utilised arguments sourced from publicly available debate platforms to chat with participants. figure 1 shows an example from the dataset. in order to evaluate open-mindedness, we follow the approach of @xcite , and ask dialogue participants whether they believe people who hold views opposite to theirs have good reasons for their convictions. @xcite argued that people who believe their ideological opponents have good reasons for their position are more likely to believe these opponents have good morals and intellectual capabilities. therefore, we also ask participants about the intellectual capabilities and morality of people who hold views opposite to theirs. we refer to these questions as the opening up minds (oum) questions and detail them in ## related work several studies on argumentative dialogues have focused on persuasion. @xcite analysed the interactions on changemyview (cmv) forums in order to understand the features that lead to persuasion. they described the original posters on cmv as "open-minded" if they changed their original view. in contrast, in our study an "open-minded" participant becomes more accepting to the opposite view, without necessarily changing theirs. wang et al. (2019) curated a dataset of dialogues where 1 dataset available at: https://github.com/youmna-h/ oumdials category questions good reasons people who (have stance x) have good reasons for (having stance x) intellectual capabilities people who (have stance x) are -unintelligent people who (have stance x) are -irrational people who (have stance x) are -ignorant morality people who (have stance x) are -unethical people who (have stance x) are -immoral people who (have stance x) are -of bad moral character recently, knowledge-based dialogue agents have attracted much attention in order to have more engaging dialogues and avoid knowledge hallucination, a typical issue in end-to-end chat models. numerous knowledge-bases have been utilised such as imdb movie reviews @xcite or wikipedia @xcite . for instance, @xcite used the woz approach to collect dialogues where wizards use sentences from wikipedia to write their responses. these wikipedia-based datasets have later been utilised to build knowledgeable dialogue agents @xcite @xcite . nonetheless, using arguments as a knowledge-base for dialogue agents has received less attention, with exception of, for example, @xcite , who developed a chatbot to persuade participants to accept that university fees should remain the same by selecting arguments from an argument graph using cosine similarity. ## wizard of oz data collection we collect 183 dialogues, using the woz approach, where a person (a wizard) plays the role of an agent and discusses a given topic with another person (a participant). statistics of the collected dialogues are shown in the wizards we recruited 5 postgraduate students from one of the author's university student job shop (a pool of students looking for research assistant work) to act as wizards. each wizard is instructed to discuss a given topic for 15-20 minutes with a participant to help them understand the other perspective on the topic being discussed rather than change their minds. more concretely, wizards are asked to use the most appropriate argument that best fits the conversation. to assist them, an argument base about the topic of discussion (see later in this section) is made available to them. each argument is annotated with a pro or con stance relative to the topic. after a participant's turn, tf-idf scores are calculated between the participant's last utterance and each argument in the argument base, foot_0 and the 50 arguments with the highest scores are presented to the wizard to help them respond. wizards are encouraged to edit the arguments they select to make them flow more naturally with the conversation, or write their own responses from scratch if they can't find a good argument to use or want to ask questions. in order to further facilitate their task, wizards are also given a list of hedges and acknowledgments to use in their responses to make the conversation more natural and polite (e.g., "i see what you mean, but...", "it could be argued...", etc.), which previous research has found to be conducive to better conversations @xcite . the woz interface also allows the wizards to use keywords to search the whole argument base of the topic, and to filter arguments by stance (pro/con). the participants we recruited participants from prolific foot_1 . all participants are fluent in english and have a prolific acceptance rate of over 95%. participants are asked to discuss the topic freely with the wizards, writing arguments and posing questions as they wish. before the conversation, participants indicate their stance on the topic of discussion by answering whether: they are vegans (if the topic is veganism), they took at least one shot of the vaccine (if the topic is vaccination), or they voted leave or remain (if the topic is brexit). according to their stance, they are asked about the people who hold the opposite stance; in particular, they indicate how much they disagree/agree with the oum questions in ## dialogue models in this section, we describe the dialogue models for the task of opening up minds. ## evaluation we evaluate the models described in section 4 using the same setup as in section 3 but by replacing the wizards with one of the models, and limiting the chat time to 10-15 instead of 15-20 minutes as the models are much faster than the wizards. we collect 150 dialogues for each of the argu-bot and wiki-bot (60 for veganism, 45 for brexit and 45 for vaccination) and 50 dialogues for the control-bot (20 for veganism, 15 for brexit and 15 for vaccination). in the remainder of this section, we present analysis of open-mindedness and chat experience for the wizards and the dialogue models. ## conclusion we presented a dataset of argumentative dialogues for opening-up minds and showed its success in positively changing participants attitudes regarding the reasons people have for their opposing views. however, this impact was lower with regard to the morality and intellectual capabilities measures, which warrants further study to these measures. we evaluated two dialogue models: a wikipedia-based and an argument-based one and showed that while they both perform closely in terms of opening up minds, the argument-based model is more successful in providing a good chat experience.