BCause: Reducing group bias and promoting cohesive discussion in online deliberation processes through a simple and engaging online deliberation tool
Year: 2023

Abstract:
Facilitating healthy online deliberation in terms of sensemaking and collaboration of discussion participants proves extremely challenging due to a number of known negative effects of online communication on social media platforms. We start from concerns and aspirations about the use of existing online discussion systems as distilled in previous literature, we then combine them with lessons learned on design and engineering practices from our research team, to inform the design of an easy-to-use tool (BCause.app) that enables higher quality discussions than traditional social media. We describe the design of this tool, highlighting the main interaction features that distinguish it from common social media, namely: i. the low-cost argumentation structuring of the conversations with direct replies; ii. and the distinctive use of reflective feedback rather than appreciative-only feedback. We then present the results of a controlled A/B experiment in which we show that the presence of argumentative and cognitive reflective discussion elements produces better social interaction with less polarization and promotes a more cohesive discussion than common social media-like interactions.

heading: 1 Introduction
text: Deliberation is the process of careful discussion before decision, and it can be defined as the thorough dialogical assessment of the reasons for and against a measure before a decision is made. When teams are geographically distributed, decision making is made more difficult by the fact that these thorough conversations cannot happen face-to-face, with people sitting in the same room. Deliberation is therefore carried out online, with social media and online discussion technologies, that are generally limited in features, are not designed to support decision making, and often produce polarisation, division and conflict (Sunstein, 2018; Golbeck et al., 2017; Matias et al., 2015; Binder et al., 2009). This
is due to a series of well-known negative effects of online communication on social media discussion platforms such as the “echo chamber” effect and the activation of biased information dynamics (Ditto and Lopez, 1992; Taber and Lodge, 2006). Research evidence clearly indicates that people tend to select information from people who hold similar positions and support similar worldviews (Huckfeldt and Sprague, 1995; Mutz, 2001). On this account, social media companies, that seek consumers satisfaction in the first place, have designed social media platforms to recommend content on the base of similarity, popularity and agreementonly principles (France, 2017). This implies that diversity of opinions and disagreement is “by design” less likely to be found in the same social media endeavour, creating platform islands, group think and isolation effects. This combination of homophily and lack of content variety has proved to degrade the quality, balance and safety (Golbeck et al., 2017; Guntuku et al., 2017) of online discourse, up to undermining social tolerance (Mutz, 2002).
In this paper, we describe the initial design and evaluation of a new platform for online deliberation BCause1, designed to be highly usable yet enable higher quality discussions than traditional social media. We present a user study and A/B tests that show how BCause improves engagement and collaboration while reducing group bias in online discussions.

heading: 2 Background knowledge
text: Online deliberation focuses on the challenge of sustaining discourse and collaborative knowledge construction through crowdsourcing unstructured contributions to online dialogue. As a field of research, it plays a crucial role in understanding and implementing new deliberative citizenship prac-
1http://bcause.app
39
tices (Law and Urry, 2004). In its most empowering interpretations, online deliberation research and practice should be intrinsically driven by the effort to produce democratically reflective citizens and to “align with the less powerful rather than reproduce the power of the already dominant” (Coleman and Moss, 2012). Research on online deliberation is thus strongly linked to democratic values and aspirations and focuses on a bottom-up view of responsible citizenship and civic behavior (Barnett, 2003; Dean, 1999). A comprehensive review of the literature on online deliberation found that despite the diversity of definitions and applications of the term, there are six main operating principles that should guide the theory and applications of online deliberation: rationality, interactivity, equality, civility, concern for the common good, and constructive attitude (Friess and Eilders, 2015). Numerous technologies have been proposed in the literature to facilitate online deliberation capabilities, from social media to targeted participatory democracy solutions (such as Decidim2, Consul3, Loomio4, etc.). While methods and skills to facilitate online dialogue have already been proposed in the literature (Collison et al., 2000), technologies for structured and quality online dialogue are still lacking. Personal e-mail systems, chat rooms, Twitter or Facebook conversations, as well as most interfaces for deliberative democracy solutions (as the ones mentioned above) are not much different from common discussion forums, where the thread of messages follows the temporal sequence of the dialogue, without regard to the issues raised, the structure of the discussion and the relevance of those issues in the ongoing conversation. The literature on online deliberation argues that topic and issue mapping tools (Conklin, 2008) provide better virtual environments for online discussion because they keep the focus on the issue rather than the time thread (as in normal discussion forums), thus avoiding noise and improving content understanding and navigation (Klein and Iandoli, 2008). However, these tools have so far struggled to spread, mainly due to usability issues and lack of capacity to enable more intuitive and inclusive narrative forms of dialogue and deliberation.
In general there is a plethora of shortcoming with discussion occurring online. For instance, the diffi-
2https://decidim.org/ 3https://consulproject.org/en/ 4https://www.loomio.com/
culty to ensure that all participants have an equal opportunity to contribute to the discussion.(e.g. Wikipedia edits (Shaw and Hargittai, 2018). This can be especially challenging in large group discussions, where some voices may be drowned out by others (Shortall et al., 2021). Moreover, sometimes in heated debates participants get sidetracked or engage in personal attacks. This can make it difficult for the group to reach a consensus or make a well-reasoned decision (Neurauter-Kessels, 2013). Apart of organic difficulties, online discussions can also be subject to manipulation or bias where abusers heavily dominate or direct discussion (e.g. (Elyashar et al., 2017a)), as it can be difficult to verify the identity of participants or ensure that they are acting in good faith (Elyashar et al., 2017b).
Online discussion platforms can be however a powerful tool for group deliberation, it is important though, to carefully consider these potential drawbacks when using them. Specific to this study, we attend the following issues of online discussion:
• Polarization: participants become more entrenched in their positions and less willing to consider other perspectives. This can happen for a variety of reasons, such as the tendency of people to seek out information that confirms their existing beliefs, or the fact that online discussions can sometimes become heated or adversarial (Boyd, 2023). This leads to more division rather than coming to a consensus. This is usually aviated by establishing ground rules (or a protocol of interaction), heavy moderation with users with aleviated rights and explicit role to to ensure that discussion remains civil and productive, or encouraging participants to consider different perspectives (Strandberg et al., 2019). But this moderation comes with high costs and often does not allow conversations to be supported at scale. Reducing polarisation in nonmoderated platforms is an open challenge. Social media are indeed the predominant solution to un-moderated online discussions but have been demonstrated to contribute to increasing polarization either by amplifying or escalating social processes that also occur offline. (Iandoli et al., 2021).
• Shallow content: In some cases, online discussions may be quite deep and consist of wellreasoned, thought-provoking content (Hara
et al., 2000; Gilbert and Dabbagh, 2005). In the majority of the cases though, the content may be more shallow and consist of superficial or unoriginal ideas (Maurino, 2007). Ultimately, the depth of an online discussion will depend on the quality of the participants and the effort they put into contributing to the conversation (Amichai-Hamburger et al., 2016). Regardless though, to encourage deep, meaningful discussions, it can be helpful to provide clear guidelines for participation and to encourage participants to carefully consider their ideas before posting (Zhang; Wang, 2019), this still requires human moderation of the conversation and it remains an issue in unmoderated discussion systems.
• Sensemaking in online discussion can prove problematic. As large discussions can be chaotic or disorganized, it is difficult for participants to follow the conversation or understand what is being discussed (Abbas et al., 2018). Additionally, not only the discussion itself but also the large number of participants, makes it challenging for individuals to keep track of all the different ideas being discussed and their provenance (who tells what). Which is setting the premise to claim that argumentative discussion could helps towards improving participants’ sensemaking. Argumentation technologies have been shown to support sensemaking (Carstens et al., 2015) but lack engagement and uptake from a larger user community.
• Collaboration: While the promise of online discussion is a highly collaborative environment where participants are working together effectively to generate new ideas, share information, and make decisions, the reality is that is usually realised as less efficient, with participants struggling to effectively communicate and work together. Factors that can affect the quality of collaboration in online discussions (Blake and Scanlon, 2012) include the clarity of the discussion goals, the diversity of perspectives represented, and the reciprocity of communication (which enables idea refinement and common ground building).

heading: 3 Motivation
text: This motivates the design of a new online deliberation system which can be highly usable and equally
engaging than current social media while providing structure to the online conversation so to improve the quality of sensemaking and collaboration in the online discussion process.
Our main hypothesis is that the right design decisions on the structure and functionalities would benefit the quality of the deliberation itself and the sensemaking of participants in it. Commonly used social interfaces have a great impact in people’s political behaviour and decision-making in general (Lewandowsky et al., 2020) - without any design intervention they even risk aiding and abetting hateful rhetorics (Bail, 2022). Our motivation is to address the challenge for building large scale online discussion platform by exploring new user interface paradigms which combine structuring with usability thus providing powerful technologies for highly usable deliberation on the Web. To achieve this we followed an approach that combines two main innovations:
• (i) Low cost Argumentation Structuring with direct replies: we designed a highly usable UI for users to contribute structured arguments while maintaining the possibility to directly address participants to the discussion, by replying to their contributions as in a normal online conversation. By providing direct replies (often missing in argumentation technology) we hope to enable reciprocity and social interaction without losing focus on the issues and structure of the conversation. This tradeoff between structure and sociability aims to improve engagement with the conversation.
• (ii) distinctive use of reflection feedback (rather than appreciation only mechanism): to support sensemaking of participants to the discussion while reducing group think and polarisation we designed a reflection mechanism for users to focus on the key value being civic, quality democratic deliberation. Such process aims to shift participants from perceiving the debate as a winning-losing contest and focus instead on the value of collaboration, trust and evidence-based thinking.
We followed an agile development approach constituted in a series of test and learn phases in which design ideas where proposed, prototyped and quickly tested in the design team. This was consisting of two UX designers and two software engineers who specialised in argumentation technologies. In
formulating our design, we drew inspiration from established argumentation technological tools; attempting to utilise their strengths and mitigate their limitations. After several test and learn cycles, which lasted one and half year, we produced the first fully functioning interface ready for testing which we describe below.

heading: 4 Design
text: Our approach is to design a tool that considers the impact of it on society and individuals and mitigates the problematic phenomena observed in these systems. This is in alignment with Value Sensitive Design (VSD) (Friedman et al., 2002) approach of supporting human values and promoting social justice.
Following a kickoff meeting where we used Q-Methodology adapted for HCI (O’Leary et al., 2013), we set a list of aspirations and fears of our designers and engineers. After establishing a theoretical foundation for the values and principles that would guide our platform’s design, we initiated the development process, utilizing iterative design sprints (Banfield et al., 2015). A number of the design aspirations identified require systemic organizational actions. Such actions are but are not limited to, the facilitation of diverse modalities of online dialogue, such as informal and goal-oriented discussions, the integration of collective decisionmaking techniques within business or enterprise workflows, and the development of an agile system that can be readily adapted to meet community requirements. While those go beyond system design, other guidelines can be followed by making design choices in terms of UX/UI. For instance design processes that allow users to inspect, confirm, dispute and correct past conversations, facilitate transparency, especially in key pieces of information processes, avoid pure argument-centric solutions, employ hybrid interfaces that retain time order and loosely visualize argument structures, are some candidate solutions. This process helped to elicit users’ perspectives and finally deduce the following design interventions:
• Argument-centric structure of discussion. We organise the deliberation as tree structure made up of debate topics (issue to be discussed), positions (opinions or possible solutions to the topic imposed), and arguments (statements that support (pro) or oppose (con)
the parent position), see Figure 1 This follows the well known paradigm of IBIS system (Kunz and Rittel, 1970; Walton, 2005) and it has many advantages such as better signalto-noise ratio, logical structure, implicit encouragement to support with hard evidence, and others, but is not widely adopted as it is considered difficult to integrate in scale and is thought to require skillful information mappers, and enables limited participation.
• Agreement slider: Before entering a pro or con argument, a user is asked to enter their level of support or disagreement to the given position (ranging from “Strongly disagree” to “Strongly agree”, see Figure 4. This is a gentle implicit “nudge” to reflect and state their agreement before supporting/refuting it with a concrete argument. In the end, he is shown the collective distribution of the group agreements on this position.
• Reflection card: We identified four important reflection dimensions: trustworthiness (of the information given in the position), whether the position is polarized, whether it should be prioritized and prediction of the group agreement on it, see Figure 2. In the end, their reflection is visualised in a radial chart along with the community’s average - to provoke a comparison to the “crowd” mean. Together with agreement slider, they are considered nuanced reflective feedback elements (not only appreciative-only as “like”/“thank you”).
• Reply functionality: a reply button enables to directly address a position or argument - without entering an additional position, see Figure 3 This helps to incorporate additional semantic information and scope user’s action context.

heading: 5 Research question
text: Our main hypothesis is that the right design decisions on the structure and functionalities, along with efficient incorporation of computational tools in online deliberation platforms would benefit the quality of the deliberation itself and the sensemaking of participants in it. Our motivation is to address the challenge for building large scale online discussion platform while balancing a critical tension between providing advanced computational
services, versus permitting people to make contributions with very little useful indexing or structure. Furthermore, since large scale discussions are hard to monitor and make sense of, our visual interfaces will be tailored to make sense and assess the state, progress and quality of a deliberation process. Towards this we aimed to explore new user interface paradigms to build usable but powerful technologies for highly usable deliberation on the Web.
To test our two main design solutions to the points i, ii above, we designed and focused our experiment to answer the following research questions:
RQ1: To what extent reflective feedback can improve engagement and sensemaking while reducing polarisation in argumentation-based discussions compared to appreciative feedback-only solutions?
RQ2: Can direct replies improve engagement but still avoiding polarisation, in argumentationbased discussions? And what is their effect on participants’ engagement?

heading: 6 Methodology
text: To test those, we carried out an controlled A/B experiment with the four conditions:
• Condition A: this design variation contains a stripped down baseline - does not contain any of the agreement or reply buttons. It resembles a typical messaging platform (e.g. WhatsApp) where posts occur chronologically with no argumentation structure with also typical appreciative only feedback functionalities (“like” and “thank” you buttons).
• Condition B: In this design variation users’ posts are organised in an argumentative fashion following an IBIS (Kunz and Rittel, 1970) approach, i.e. organising posts as positions and supporting or opposing arguments. The appreciative only feedback functionalities are retained.
• Condition C: Is an extension of condition B with more nuanced positive/negative feedback elements: i. agreement slider, ii. reflection feedback cards. From the appreciative only feedback elements we retain only the “like” button.
• Condition D: A full-fledged version containing all the elements of previous conditions (
argumentative structuring, nuanced feedback) but also direct reply fucntionality.

heading: 6.1 Evaluation factors
text: We evaluated each condition against sensemaking and engagement factors. The chosen sensemaking features we used are an extract of Alsufiani et al. (2017) work on deducing theoretical features of Sensemaking, with an extra feature to assess Reflection (as defined by Weick (1995)) and proposed by De Liddo et al. (2021). Engagement factors are derived from O’Brien and Toms (2010) with adaptation to online discussion. Both are shown in Table 1 along with the question prompt given to crowdworkers. Note that some questions are given in negative form - this was later reversed in the analysis.

heading: 6.2 Experiment design
text: Each condition was tested in a group of 18 participants. To ensure limited bias we repeated the same trial three (3) times. In total, we recruited 216 participants (18 participants x 4 conditions x 3 trials). Users were recruited via Amazon Mechanical Turk and offered a compensation of 10$ per hour. We compared the discussion UI (3 different versions of it with argumentation structuring plus appreciative only feedbacks (condition B), structuring with nuanced positive/negative feedback (agreement slider plus reflection feedback cards (condition C), and full-fledged version (with structuring, nuanced feedback and reply (condition D)) against a “Whatsapp” like unthreaded discussion interface (condition A - used as a baseline). Participants were asked to contribute to a discussion that was pre-populated with 6 posts (in case of the argumentative conditions, 3 positions and 3 arguments). Within the group of 18, users could use other users’ contributions as they were happening. For a task to be considered successful, at least 2 distinct contributions were expected (positions or arguments). Users were handed a post-hoc questionnaire with the questions presented in Table 1 in a 5-level Likert scale upon completion of the task.

heading: 7 Results
text: We present in Figure 5 the descriptive statistics of the 20 variables about engagement (11 factors) and sensemaking (9 factors) of the 3 design variations along with the control variation (group A). The box error plot data shows the average values of 3 separate trials of 18 participants each.
To test variance homogeneity between the three separate 18-big batches, we ran a Levene’s test (Gastwirth et al., 2009), which showed equal variance among the three samples. We proceeded then analysing all three batches in one unified 54-big sample. Upon affirming the normality of the data distribution through the application of the ShapiroWilk test ((Shapiro and Wilk, 1965)), we proceeded with ANOVA one-way analysis (Fisher, 1992), followed up by multiple pairwise comparisons employing Tukey’s HSD ((Abdi and Williams, 2010)). The ANOVA analysis revealed no statistically significant differences among the factors, which was anticipated given the stringent Bonferroni correction (a = 0.05/6) applied to account for multiple comparisons. Nonetheless, it is noteworthy to mention that there was no degradation observed in the levels of Sensemaking and Engagement, a promising indication that the two interventions scrutinized (argument-structuring and reflective feedback elements) did not introduce cognitive load (information overload). The absence of an information overload due to the interventions as you could possibly expect, is reassuring for the seamless incorporation of those elements.
We then carried out a Social Network Analysis (SNA) on the interactions graph of each condition and evaluated network metrics, see results in Table 2. Social network analysis (SNA) can be employed in studies with a relatively small number of participants, like this one (n=18, 3 trials), particularly if interactions among the participants are expected to be complex and significantly interconnected. Even in such a small network, SNA can still provide valuable insights into the structure and properties
of the network, such as the measurement of network fragmentation (Hanneman and Riddle, 2005). We observe that conditions C and D perform better in terms of average node degree and density. Average node degree is a good indication of Social Interaction coverage, basically how well the social interactions is distributed across the group. This indicates that the presence of argumentative discussion and cognitive reflection, produces a better social interaction. Network density is measure of the connectedness of the network in terms of total number of connections divided by the maximum possible number (of the perfectly interconnected graph) - so higher density means more interconnectedness.
Transitivity also slightly improves in conditions C and D. This means that the overall probability for the network to have adjacent nodes interconnected is higher, thus revealing the existence of more tightly connected communities. Transitivity number reflects the likelihood that the network’s
nodes form interconnected triads ((Opsahl, 2013)); which is interrelated to reciprocity (the tendency of pairs of nodes to be mutually linked). The comparatively large transitivity number posits an enhanced level of reciprocal engagement amongst users. Comparing the conditions with argumentative discussion (B,C,D) against condition A we also observe better connectiveness.
Number of components is the number of subgroups or tribes. Insertion of the argumentation structuring considerably reduce platform island (reduction of 2/3). The dimension of the largest component grows and reaches almost 89 percent of the total graph. Considering the largest component size metric combined with the largest component diameter, we can say that participants organised themselves around a large centric group rather than scatter to small isolated groups. This is a strong indication that argumentative discussion reduces the phenomenon of irregularities (islands of discussion) and promotes cohesive discussion.

heading: 8 Discussion
text: Though certain pathogens of democratic dialogue are not sourced in the implementation or design technotropy of social media or other discussion
platforms but rather a certain reflection of the same problems in the virtual online environment, they are exacerbated within these platforms. Therefore complementary to technological solutions for democratic-aware design, ultimately the media and social literacy shall be pursued to address those problems. However in the interim and complementary, appropriate design solution for online discussion platforms should not be left unchecked and unaddressed.
For that we demonstrated that even seemingly small design decisions, have a significant impact on the dynamics of the discussion. Specifically, we showed that argumentation structuring is a viable and effective solution to many shortcomings of current social media technologies in supporting online deliberation. This is in line with previous research findings that structuring debates around issues and nudging participants have a positive effect, with no significant engagement drop (Tanasijevic and B"ohm, 2016).
Second, we showed that reflective appreciative feedback elements and reply interaction, equally abate some of the deficiencies of modern discussion platforms.
Overall our study findings inspire for a rebirth of
argument-centric solutions. After that we have observed an “argumentative winter” -if is permissible to employ this terminology- where people exibit distrust to argumentative technologies mainly because of the steep learning curve and low level engagement, we exhibit a solution that is comparable to typical social media, viable, feasible with equal if not better levels of engagement. This instils optimism of future solutions that will enable healthier and more civil deliberation.
Limitations
We recognise that our study has limitations. Firstly, the platform lacks elements that modern social media platform users take for granted, e.g. notifications, direct messaging, etc. Also, it was by design the absence of any moderation mechanism as we wanted to inspire a wide spectrum of opinions, even the extreme hyper-partisan views you would expect in an open platform (Oltmann et al., 2022). Further, the experiment executed to confirm our hypotheses was carried in a controlled environment with a predetermined interaction. We would expect that if reproduced in an open-ended environment several other phenomena stemming from network size, cold-start problems, user inertia or lack of trust would occur. However even though not a naturalistic setup, controlled experiment remains the best scientific device to establish a causal relationship between the examined variable and the user observed behaviour (Kohavi et al., 2007).

heading: Acknowledgements
text: This research was funded by the U.S. Office of Naval Research under award number N00014-191-2366

heading: Future Work
text: This instils optimism of future solutions that will enable healthier and more civil deliberation. Limitations
We recognise that our study has limitations. Firstly, the platform lacks elements that modern social media platform users take for granted, e. g. notifications, direct messaging, etc. Also, it was by design the absence of any moderation mechanism as we wanted to inspire a wide spectrum of opinions, even the extreme hyper-partisan views you would expect in an open platform (Oltmann et al. , 2022). Further, the experiment executed to confirm our hypotheses was carried in a controlled environment with a predetermined interaction. We would expect that if reproduced in an open-ended environment several other phenomena stemming from network size, cold-start problems, user inertia or lack of trust would occur. However even though not a naturalistic setup, controlled experiment remains the best scientific device to establish a causal relationship between the examined variable and the user observed behaviour (Kohavi et al. , 2007)