Do Differences in Values Influence Disagreements in Online Discussions? (2023)

Abstract: Disagreements are common in online discussions. Disagreement may foster collaboration and improve the quality of a discussion under some conditions. Although there exist methods for recognizing disagreement, a deeper understanding of factors that influence disagreement is lacking in the literature. We investigate a hypothesis that differences in personal values are indicative of disagreement in online discussions. We show how state-of-the-art models can be used for estimating values in online discussions and how the estimated values can be aggregated into value profiles. We evaluate the estimated value profiles based on human-annotated agreement labels. We find that the dissimilarity of value profiles correlates with disagreement in specific cases. We also find that including value information in agreement prediction improves performance.

## introduction a large number of users participate in online deliberations on societal issues such as climate change @xcite and vaccination hesitancy @xcite . disagreement is an important aspect of a deliberation @xcite since it can (1) drive novel ideas, (2) incentivize evaluation of the proposed ideas, (3) avoid echo chambers, and (4) cancel out individual biases @xcite . discussions with disagreement help users understand the opposing viewpoints @xcite . further, discussions having adequate disagreement have been associated with a higher quality deliberation @xcite . ensuring that participants express a sufficient level of disagreement in a discussion is hard. we do not know the nature of disagreement in online platforms @xcite . further, questions arise on how to control for disagreement to enhance reciprocity @xcite , and how too much exposure to opposing views drives polarization @xcite . analysis methods for online discussions currently cannot accurately represent such diverse perspectives @xcite , and measuring deliberative quality is an open challenge @xcite . we want to ensure that a discussion incorporates many perspectives and that those are actively communicated. for this reason, we turn to value conflicts, a potential root cause for disagreement. we consider the hypothesis that when users with conflicting values engage in a discussion, diverging views come up. perspective and value clashes are at the heart of disagreement @xcite . in collaborative teams, value conflicts are linked to disagreement @xcite . specifically, values are said to be an effective way to make conflict explicit among participants in a discussion @xcite . to evaluate our hypothesis, we construct value profiles based on user comments on reddit, a social media platform. a value profile captures the relative importance a user ascribes to values. we employ ten values, e.g., stimulation, universalism, and security, from the well-known schwartz theory of basic values @xcite . then, we compare the similarities among profiles to the disagreement among users on different topics. this allows us to investigate the association between value conflict (low similarity) and disagreement. figure 1 shows an overview of our approach. we gather 11.4m comments from 19k users on reddit to construct value profiles. we perform up to 200 tests with different settings to investigate our hypothesis. we further experiment with replacing estimated value profiles with self-reported ones. to do so, we collect 572 judgments from 26 annotators in combination with self-reported value profiles. selecting conversation partners based on their profile to manage value conflicts and influence the level of disagreement in a discussion could be a tool for moderators to balance conversations. to provide support for moderators, we investigate the impact of adding profile information to the agreement analysis task @xcite . since the contextual implications of values are usually unknown, connecting user concerns to values @xcite opens up human-machine collaboration opportunities for a more constructive conversation @xcite @xcite . contributions (1) we experiment with methods for value estimation from text to obtain value profiles from an online discourse (reddit comments). ## related work although there is existing work on analyzing agreement in online discussions, very few works focus on examining the reasons for disagreement. we review the existing work on agreement analysis, introduce two popular value theories, and outline previous research on value estimation. ## method figure 1 shows an overview of our method. we collect posts from users in online discussions. using a trained value estimation model, we aggregate predictions over the collection to form a value profile. then, to evaluate our hypothesis, we compare the value profiles for users known to be in disagreement based on an existing dataset. our code foot_0 and data @xcite ) is available online. ## conclusion our results on the role of value conflicts in disagreements are mixed. on the one hand, we mostly note negative evidence of a correlation between profile similarity and disagreeing users when using the vpe methods. when using self-reported profiles, the negative evidence reduces and results become inconclusive for a majority of the cases. this suggests that the nature of the profiles differs, and further investigation is necessary. on the other hand, we observe that value conflicts were found to lead to disagreements in specific cases. when values are likely to be relevant and diverse, we find evidence for a correlation between value conflict and disagreement. while value conflicts may not be directly related to disagreement, they do signal diversity with respect to the underlying motivations of participants. using value profiles in combination with bert performs marginally better than a text-only baseline in predicting agreement. yet, vpe can be valuable for characterizing and enhancing diversity in discussions. further, making participants value-aware could enhance the discussion quality. constructing profiles from behavioral cues, such as written opinions, is noisy. for future work, we hope to see the creation of resources that allow end-to-end evaluation by combining text posts with a consistent set of users that allows aggregation to ground truth profiles or self-reported profiles. however, gathering such profile information outside controlled lab settings is highly complex. future experiments may incorporate more judgments and provide stronger evidence for one hypothesis. these can be retrofitted with our results through bayesian updating @xcite .