Towards Breaking the Self-imposed Filter Bubble in Argumentative Dialogues (2023)

Abstract: Human users tend to selectively ignore information that contradicts their pre-existing beliefs or opinions in their process of information seeking. These “self-imposed filter bubbles” (SFB) pose a significant challenge for cooperative argumentative dialogue systems aiming to build an unbiased opinion and a better understanding of the topic at hand. To address this issue, we develop a strategy for overcoming users’ SFB within the course of the interaction. By continuously modeling the user’s position in relation to the SFB, we are able to identify the respective arguments which maximize the probability to get outside the SFB and present them to the user. We implemented this approach in an argumentative dialogue system and evaluated in a laboratory user study with 60 participants to show its validity and applicability. The findings suggest that the strategy was successful in breaking users’ SFBs and promoting a more reflective and comprehensive discussion of the topic.

## introduction spoken dialogue systems are getting increasingly popular, especially as they enable easy access to requested information from online sources, such as search engines or social media platforms. specifically, with regard to more complex interactions, two important phenomena can be observed that can result in information bias. on the one hand, due to filter algorithms, information content is selected based on previous online behavior, which leads to cultural/ideological bubbles, the so-called "filter bubbles" @xcite . on the other hand, nickerson (1998) points out that users who are confronted with controversial topics tend to focus on a "biased subset of sources that repeat or strengthen an already established or convenient opinion." this user behavior leads to the so-called "self-imposed filter bubbles" (sfb) @xcite and "echo chambers" @xcite @xcite . both are manifestations of "confirmation bias", a term typically used in psychological literature. these phenomena are mutually dependent according to @xcite as the sfb is reinforced and perpetuated due to algorithmic filters delivering content aligned with presumed interests based on search histories. moreover, @xcite claim that studies have shown that individual choice has even more of an effect on exposure to differing perspectives than "algorithmic curation". in this paper, we focus on the second phenomenon, namely the user's sfb regarding a controversial topic during the interaction with an argumentative dialogue system (ads). building upon the work of @xcite , we model the user's sfb using the following four main dimensions: reflective user engagement (rue), personal relevance (pr), true knowledge (tk) and false knowledge (fk). the concept of rue encapsulates the user's critical thinking, building upon the definition established in our prior work @xcite . on the other hand, pr pertains to the individual user's assessment of the significance of subtopics, further on called "clusters", in relation to the overarching topic of discussion. true knowledge (tk) is characterized as the information already possessed by the user on a particular topic. conversely, false knowledge (fk) entails the user's false beliefs and misinformation on the respective topic. based upon these dimensions we have the ability to construct a model for assessing the likelihood of a user being caught within an sfb. in order to achieve this, we ascertain the user's position along these four dimensions and consistently update it throughout the course of the dialogue. building upon sfb-model we 1) introduce a rule-based system policy to break the user's sfb during an ongoing interaction and 2) validate our policy in a laboratory study by comparing it to a user-interest-driven system policy. the remainder of this paper is as follows: section 2 gives an overview of related literature, followed by a description of the underlying sfb-model and our proposed rule-based sfb-breaking policy in section 3. section 4 discusses an exemplary integration of our model/policy in an ads, which is evaluated in a laboratory study described in section 5. section 6 covers the respective study results, followed by a discussion of the former and study limitations in sections 6 and 8. we close with a conclusion and a brief discussion of future work in section 9. ## related work in the following, we provide a brief overview of the existing literature on the main aspects of the work presented herein, confirmation bias and selfimposed filter bubbles and argumentative dialogue systems. ## self-imposed filter bubble model in the following section we will give a short overview on the sfb-model we adapted to and its respective dimensions. this serves as a basis for our system's sfb-breaking policy introduced in subsection 3.3. ## sfb-model and policy integration into the ads in the following, the relevant components of the ads, namely the knowledge base and dialogue model, focusing on the exemplary integration of our sfb-model. in order to combine the presented model with existing argument mining approaches, ensuring its adaptability with respect to discussed topics, we adhere to the bipolar argument annotation scheme introduced @xcite foot_1 . this scheme encompasses argument components (nodes), structured in the form of bipolar argumentation trees. the overall topic represents the root node in the graph. we consider two relationships between these nodes: support or attack. each component, excluding the root node (which has no re-lation), has exactly one unique relation to another component. this results in a non-cyclic tree structure, wherein each node, or "parent", is supported or attacked by its "children". if no children exist, the node is a leaf and marks the end of a branch. furthermore, the sfb-model necessitates semantically clustered arguments, wherein each argument pertains to one or more clusters related to the discussed topic. given that an argument can encompass multiple aspects of a topic, it may belong to several overlapping clusters @xcite . every argument directly addresses one or more clusters. since each argument component targets the preceding parent, it indirectly refers to all preceding parents. consequently, we stipulate that each argument component inherits the clusters of its preceding nodes, meaning it indirectly encompasses all clusters that its parent addresses, whether directly or indirectly. notably, the root node is not affiliated with a cluster. in this ads, a sample debate on the topic marriage is an outdated institution provides a suitable manually clustered argument structure. it serves as the knowledge base for the arguments and is sourced from the debatabase of the idebate.org foot_2 website. it consists of a total of 72 argument components, their corresponding relations, and is encoded in an owl ontology @xcite for further use. in each whypro/con move, a single supporting/attacking argument component is presented to the user. to prevent the user from being overwhelmed by the amount of information, the available arguments are presented to the users incrementally upon their request. in order to integrate the sfb-model 3.2, the dialogue model has to provide respective user moves. the interaction between the system and the user is separated into turns, consisting of a user action and the corresponding natural language answer from the system. the system's response is based on the original textual representation of the argument components, which is embedded in moderating utterances. as shown in ## user study we conducted a user study from october 4 th to 15 th , 2022, involving 60 participants. the participants were divided into two groups: one group was presented with arguments based on their interests (referred to as the "interest" group), whereas the other group was presented with arguments that might challenge their existing beliefs (referred to as the "sfb-breaking" group). in the interest group, the system presented arguments that precisely matched the user's requests. if a loss of interest was detected (modeled by an interest model ( aicher et al., 2022a )), the system suggested arguments that aligned best with the user's preferences and interests. this interest policy is based on our previously introduced interest model @xcite and adapted accordingly. in the sfbbreaking group, the system presented arguments based on the system policy described in subsection 3.3. consequently, the arguments presented to the sfb-breaking group might have differed in polarity and/or cluster from the original user request. the primary objective of this study was to address the following research questions: 1) can the proposed system policy effectively break a user's sfb? 2) what are the discernible differences in the overall sfb dimensions between the two participant groups? to investigate these research questions, we formulated the following hypotheses to be tested during the study: h1 participants in the sfb-breaking (interest) group exhibit a lower (higher) probability of being caught in an sfb after the interaction. h2 the exploration behavior of the sfb-breaking group changed during the interaction. these hypotheses were designed to assess the effectiveness of the system policy in breaking the users' sfbs and to explore potential differences in sfb dimensions between the two groups. the study was conducted in a laboratory setting at a university, involving international participants who possessed a sufficient level of proficiency in english. including the introductory phase and the completion of pre-and post-questionnaires, the entire study duration was estimated to be one hour. participants were compensated with a payment of 10@xmath0/hour. after a brief introduction to the system, including a short text and instructions on how to interact with it, participants were required to answer two control questions. these questions served as a means to verify their understanding of how to interact with the system. only participants who successfully passed this test were allowed to proceed to a test interaction with the system. during the "real" interaction, participants were instructed to listen to at least 20 arguments foot_4 . participants were not informed about the underlying sfb or interest model. they were only informed that the ads might provide suggestions on its own, and they could return to the previous argument if they did not approve. throughout the study, the following data was collected: self-assessment questionnaire (p.851, 2003), calculated sfb-values: rue, pr, tk, and fk (for each cluster k), participants' opinions and interests regarding the topic of discussion, set of heard arguments, dialogue history. strict adherence to data protection regulations and participant anonymity was maintained throughout the study. participants had the freedom to withdraw from the study at any time. the study was approved by an institutional review board (irb) after thorough ethical review and met all internal guidelines due to the solely cooperative, non-persuasive design of the user study. ## results the user study involved 60 participants, ranging in age from 22 to 41 years. the participants' average age was 28.45 (with a standard deviation (sd) of 4.11). the two participant groups each consisted of 30 individuals (sfb-breaking: 7 females, 23 males; interest: 10 females, 20 males). both groups exhibited similar levels of experience with spoken dialogue systems, rated on a 5-point likert scale where 1 represented "no experience" and 5 represented "very much experience": interest group at 2.40 (sd 0.89); sfb-breaking group at 2.13 (sd 1.04). on average, participants spent approximately 33.87 minutes engaged in interactions with the system (interest group: 33.99 min (sd 7.74), sfbbreaking group: 33.75 min (sd 5.96)). throughout the interaction, participants were presented with an average of 22.02 arguments (interest group: 21.73 (sd 4.00), sfb-breaking group: 22.30 (sd 3.54)). regarding the "pre-interest" of the participants (measured on a 5-point likert scale before the interaction, where 1 represented "not at all interested" and 5 represented "very much interested"), the difference between the two groups is insignificant (interest: 3.67 [sd 0.71], sfb-breaking: 3.47 [sd 0.82]; p @xmath1.986). similarly, the difference in their "pre-opinion" (rated on a scale of 1 to 5, where 1 represented "totally disagree" and 5 represented "totally agree") is also insignificant (interest: 3.09 [sd 0.93]; sfb-breaking: 2.78 [sd 0.83]; p @xmath2.308). during the interaction, approximately 36.67% (11 out of 30) participants changed their opinion (from pro to con or vice versa) in the sfbbreaking group, compared to 6.67% (2 out of 30) in the interest group. regarding the "post-interest" (measured after the interaction), a significant difference with p @xmath3.024 < 0.05 = α is notable (interest: 3.20 [sd 1.16], sfb-breaking: 3.97 [sd 0.89]). similarly, the "post-opinion" also exhibits a significant difference (interest: 3.63 [sd 0.96], sfb-breaking: 3.07 [sd 0.87], p @xmath4.048, @xmath5.29). to determine the significance of the difference between pre-and post-measurements, we utilized the non-parametric wilcoxon signed rank test @xcite for paired samples and bonferroni corrected p-values p corr based on a set of four comparisons. for the sfb-breaking group, both interest and opinion showed significant differences before and after the interaction (interest: p @xmath6.006, @xmath7.38; opinion: p @xmath8.036, @xmath9.30). in the interest group, the pre-and post-interest also exhibited significant differences (p @xmath10.006, @xmath11.39). considering the user moves, a significant difference between both groups becomes evident. in the interest group, a pro (con) argument was requested 297 (172) times. only in 15% of all argument requests, interest group users asked for an argument which did not align with their own opinion. in the sfb-breaking group, a con (pro) argument was requested 117 (90) times. furthermore, in 71 (82) instances, the ads opted to present a con (pro) argument. particularly towards the end, the sfbbreaking group tended to request arguments without specifying polarity, and if polarity was specified, it contradicted the user's opinion in 43% of all requests. in the interest group, arguments were rarely rejected (3) and mostly preferred (87). in the sfb-breaking group, suggested arguments were rejected 65 times and explicitly preferred 71 times. moreover, participants in the sfb-breaking (interest) group requested to return to the previous argument in only 8 (1) cases. ## discussion in the following the results of our study (section 6), particularly regarding our two hypotheses (refer to section 5) are discussed. ## limitations however, this work has certain limitations that could be addressed in future research. first, the sample size of our study is relatively small, potentially affecting the generalizability of our findings. in future endeavors, a study (e.g., through crowdsourcing) with a larger sample size could yield more robust data, enabling us to refine the sfb margins and enhance the validity of our approach. second, given that the sfb-model is a novel concept, it is presently constrained to four dimensions. subsequent research could explore additional dimensions that may prove pertinent in various scenarios and applications. additionally, finding ways to implicitly estimate both pr and tk, which can only be determined retrospectively, would be advantageous. this could involve leveraging common sense knowledge bases and employing fake news detection techniques. third, while our study demonstrates the proof-of-principle for the effectiveness of a rule-based policy to break sfb, it is limited to static, predefined rules, rendering it relatively inflexible. in future work, we intend to delve into more advanced machine learning techniques, such as reinforcement learning. this would enable us to personalize and adapt these strategies based on the user's verbal and non-verbal feedback, thereby ensuring the user's satisfaction and sustaining their willingness to engage in the dialogue. ## conclusion and future work in this work, to the best of our knowledge, we introduce a novel approach to break the user's sfb. after shortly explaining the underlying sfb-model, we define a rule-based system policy to break the respective user sfb during a cooperative dialogue with an argumentative dialogue system and validate it in a laboratory user study. the study results strongly indicate the effectiveness of the proposed system policy in reducing the likelihood of being stuck in an sfb compared to a policy that prioritizes the users' greatest interest. moreover, the study revealed significant changes in users' exploration behaviors during the interaction. in particular, the sfb-breaking participants requested arguments of both polarities almost equally often after the ads pointed out that the previous exploration seemed to be one-sided. these findings emphasize the influence of the system policy on users' exploration behaviors and opinions, further highlighting the success of the proposed approach in mitigating sfb tendencies and fostering open-mindedness in an argumentative dialogue. in future research, we will augment our system's policy by incorporating sophisticated techniques for perceiving and interpreting the user's non-verbal social signals (gestures, facial expressions) in real-time during the interaction. building upon estimation methods for sentiment and emotion recognition, we aim to leverage reinforcement learning to optimize the system's policy, enabling it to dynamically adapt to each individual user's motivation and effectively engaging the users to recognize and overcome their sfb. in conclusion, this paper highlights the importance of addressing sfbs in argumentative dialogues and takes us a step closer to enabling users to build a well-founded opinion and foster critical, reflective thinking, and open-mindedness in their interaction with cooperative ads.